| programmerjake | They appear to have requireMatrixJoined set to true for #libera, which causes tge bridge to perpetually stop relaying messages since the irc/matrix user lists are perpetually out of sync due to the sheer volume of users leaving/joining | 01:17 |
|---|---|---|
| * lkcl waves to sasi8985 | 13:18 | |
| lkcl | sasi8985: apologies earlier for recommending that you pursue RISC-V. i explained to jacob that you're here because of Ganesan's IBM Power ISA Programme | 13:20 |
| lkcl | whilst technically correct that RISC-V is "easier", well... here is some reading material: | 13:20 |
| lkcl | https://news.ycombinator.com/item?id=24459314 | 13:20 |
| sasi8985 | No problem i want know development the flow from rtl to os...so i thought i will be starting with simple things | 13:21 |
| sasi8985 | since microwatt is in vhdl 2008 some fatures are not supported in vivado....so i am thinking to write rtl for power isa to sharpen my skills | 13:23 |
| sasi8985 | by rtl i mean in verilog.. | 13:23 |
| sasi8985 | https://github.com/antonblanchard/microwatt/issues/293 | 13:24 |
| lkcl | you can use GHDL and yosys with the yosys-ghdl plugin to convert microwatt to verilog | 13:24 |
| lkcl | this has been done a number of times, you can if you search online find pre-converted verilog HDL for microwatt | 13:25 |
| sasi8985 | yeah i found it...but it is not understandable | 13:26 |
| sasi8985 | https://git.libre-soc.org/?p=libresoc-litex.git;a=blob;f=microwatt/microwatt.v;h=87dfc352079465813f14c43bb8c0069a52a6b596;hb=6efd2e59703f6f0747435f97030e8a463233457f | 13:26 |
| lkcl | indeed. so you can learn to read VHDL. it took me only a few weeks. | 13:26 |
| lkcl | i still can't _write_ VHDL, but i can at least read it :) | 13:27 |
| lkcl | the difference between := and = was a bit of a pain, but i got there in the end | 13:27 |
| lkcl | the other one is A2O which *is* in verilog | 13:27 |
| lkcl | and ChiselWatt is amazingly readable (and incredibly short) | 13:27 |
| lkcl | Anton's a software engineer, so knows the value of writing readable code and including comments | 13:28 |
| sasi8985 | okay...i will see it...but also i want to write on my own....for that i need to learn some things and there are lots of gaps in my knowledge | 13:29 |
| sasi8985 | instead of following traditional rtl can i go with systemc as we can improve performance as well? | 13:47 |
| lkcl | ghostmansd|2, hello this is lkcl | 16:51 |
| ghostmansd|2 | hello! | 16:51 |
| *** ghostmansd|2 is now known as ghostmansd | 16:52 | |
| lkcl | sasi8985: systemc is a proprietary tool. Libre-SOC doesn't use proprietary tools (you are entirely free to choose whatever you want of course) because they could contain spying backdoors | 16:52 |
| lkcl | ghostmansd: so generally, the rule for IRC is: just write the question and leave the IRC client running 24x7 | 16:52 |
| lkcl | this way, you at least "eventually" complete a conversation :) | 16:53 |
| ghostmansd | :-) | 16:53 |
| lkcl | we have IRC logs here if you ever get disconnected | 16:53 |
| ghostmansd | is there some client recommended for Android? | 16:53 |
| lkcl | https://libre-soc.org/irclog/index.html | 16:53 |
| lkcl | i tried one by installing F-Droid | 16:53 |
| lkcl | then searching in there | 16:54 |
| lkcl | it sucked :) | 16:54 |
| lkcl | YMMV | 16:54 |
| ghostmansd | no doubts, it's Android, after all :-D | 16:54 |
| lkcl | we're bridged over to here https://chat.openpower.foundation/opf/channels/libre-soc | 16:54 |
| lkcl | thanks to toshaan | 16:54 |
| lkcl | which may be easier (it relies on the openpowerbot being active though, and sometimes it dies) | 16:55 |
| lkcl | anyway, we're both here :) | 16:55 |
| lkcl | btw did you run this script for setting up the repos? https://git.libre-soc.org/?p=dev-env-setup.git;a=blob;f=hdl-dev-repos;hb=HEAD | 16:56 |
| lkcl | because that installs c4m-jtag for you | 16:57 |
| ghostmansd | I think I followed manually via instructions from https://libre-soc.org/HDL_workflow/#index9h1 | 16:58 |
| ghostmansd | let me check the script and try using it | 16:58 |
| lkcl | the important thing is to use the "git tag | xargs blah blah" | 16:58 |
| lkcl | f****g f****g python setuptools-scm gets in the way. sigh | 16:59 |
| lkcl | there is a workaround there: define some ENV VAR which tells setuptools-scm to stop trying to be clever | 16:59 |
| lkcl | and use an explicit revision tag (defined in the ENV VAR) rather than inspect the git tags and get it hopelessly wrong | 17:00 |
| lkcl | if you've already got some repos set up we can run a few tests and see what happens | 17:00 |
| lkcl | the primary one is openpower-isa | 17:00 |
| lkcl | can we please check that by running some unit tests (rather than re-cloning and waiting for that to complete)? | 17:01 |
| ghostmansd | it's already cloned, the script is doing some stuff in the background, though | 17:01 |
| ghostmansd | but let's check | 17:01 |
| lkcl | excellent | 17:01 |
| lkcl | ok make sure you're in this directory: | 17:02 |
| ghostmansd | I'm at openpower-isa | 17:02 |
| lkcl | lkcl@fizzy:~/src/libresoc/openpower-isa/src/openpower$ | 17:02 |
| lkcl | can you check first run "python3 --version" for me? | 17:02 |
| lkcl | should be "Python 3.7.2" (or close) | 17:02 |
| ghostmansd | Python 3.7.3 | 17:03 |
| lkcl | excellent | 17:03 |
| lkcl | ok try this: | 17:03 |
| ghostmansd | even more than close :-) | 17:03 |
| lkcl | python3 decoder/isa/test_caller.py > /tmp/f | 17:03 |
| lkcl | this runs quite a lot of tests | 17:03 |
| lkcl | ...s.................. | 17:04 |
| lkcl | ---------------------------------------------------------------------- | 17:04 |
| lkcl | Ran 22 tests in 54.213s | 17:04 |
| lkcl | OK (skipped=1) | 17:04 |
| ghostmansd | well, it's started | 17:04 |
| lkcl | that's a good sign :) | 17:04 |
| lkcl | it's reaaaally verbose btw, hence why pipeline to a file | 17:04 |
| lkcl | getting dots, no "F" or "E" ? | 17:05 |
| ghostmansd | 22 tests, 1 skipped, and a huge neat deprecation warning :-) | 17:05 |
| lkcl | hurrah! | 17:05 |
| lkcl | that's fantastic! | 17:05 |
| lkcl | congratulations, you just ran a bunch of simulated Power ISA instructions | 17:05 |
| lkcl | and they didn't break | 17:05 |
| ghostmansd | superb! | 17:05 |
| ghostmansd | 7.4M of logs | 17:06 |
| ghostmansd | my respect! | 17:06 |
| lkcl | i know... :) | 17:06 |
| lkcl | ok so let's do some experimenting / messing about | 17:06 |
| lkcl | edit that file (decoder/isa/test_caller.py) | 17:06 |
| lkcl | and do a global search/replace "def test_" with "def notst_" | 17:07 |
| lkcl | (or something which basically doesn't have "test" in the function name) | 17:07 |
| ghostmansd | s/notst_/notest/ ? | 17:07 |
| lkcl | ah notest has the word "test" in it :) | 17:07 |
| lkcl | i made that mistake lol | 17:07 |
| lkcl | tst or tt or "flibblebubble" - just anything *other* than "test" :) | 17:08 |
| lkcl | save it and then in a new console do "git diff" to make sure it's *only* the function names "def test_xxxx" that have changed to "def whatever" | 17:09 |
| ghostmansd | - def test_add(self): | 17:09 |
| ghostmansd | + def notest_add(self): | 17:09 |
| ghostmansd | and similar | 17:09 |
| ghostmansd | looks like OK, let's re-run? | 17:09 |
| lkcl | notest will still be picked up by the python unittest module | 17:09 |
| ghostmansd | this should skip some tests, right? | 17:10 |
| lkcl | it does have to *not* contain - at all - the word "test" - *anywhere* in the function name | 17:10 |
| lkcl | yes, that's what we're trying to achieve | 17:10 |
| ghostmansd | re-run the suite, 0 tests are run | 17:10 |
| lkcl | i usually use "tst" | 17:10 |
| lkcl | ah brilliant! | 17:10 |
| ghostmansd | deprecation warning is still there | 17:10 |
| lkcl | ok i will remember that. | 17:10 |
| lkcl | ok good | 17:10 |
| lkcl | now | 17:10 |
| lkcl | https://git.libre-soc.org/?p=openpower-isa.git;a=blob;f=src/openpower/decoder/isa/test_caller.py;h=edc9c2bba8f85ca331bc8f8a6bad432bb3bbdde7;hb=e4f69ca4ae12eca4f29c63742621aba5447ae013#l83 | 17:10 |
| lkcl | just after DecoderTestCase, put "test" back | 17:11 |
| lkcl | def test_add(self): | 17:11 |
| lkcl | so we only run one unit test | 17:11 |
| ghostmansd | yep, re-run Ran 1 test in 1.710s | 17:11 |
| lkcl | (much quicker, saves some time) | 17:11 |
| lkcl | excccellent | 17:11 |
| lkcl | ok, now the log output should be much shorter | 17:12 |
| lkcl | so, in the log output, search for "call add" | 17:12 |
| ghostmansd | int_op 2 | 17:12 |
| ghostmansd | call add add | 17:12 |
| ghostmansd | getitem 14 64 0x0 0 | 17:12 |
| lkcl | and at the same time (in another window) open up decoder/isa/caller.py | 17:12 |
| lkcl | brilliant. so now, in decoder/isa/caller.py search for "def call(self, name)" | 17:13 |
| lkcl | surpriise, this is the function which is "called" to execute an instruction | 17:13 |
| lkcl | this is terribly lazy programming style, btw, liberally throwing "log()" calls into the program :) | 17:14 |
| lkcl | but it's not designed for "end-user" interaction, it's designed to make it easy to see what the hell is going on when implementing research and HDL | 17:15 |
| ghostmansd | you mean `log("call", name, asmop)`? | 17:15 |
| lkcl | so do not be afraid to commit extra log() debug prints that you need to find things ok! | 17:15 |
| lkcl | yes | 17:15 |
| ghostmansd | aha, got it | 17:16 |
| lkcl | so, now you are sync'd between the log output and the code | 17:16 |
| lkcl | good | 17:16 |
| lkcl | now in the log output search "reading reg RA" | 17:16 |
| lkcl | but | 17:16 |
| lkcl | first | 17:16 |
| lkcl | note in def test_add unit test, the instruction listing is | 17:16 |
| lkcl | lst = ["add 1, 3, 2"] | 17:17 |
| lkcl | and if you look *here*: https://git.libre-soc.org/?p=openpower-isa.git;a=blob;f=openpower/isa/fixedarith.mdwn;h=bff81b428bcb3d8a37134e2c02a29a36dc36705b;hb=e4f69ca4ae12eca4f29c63742621aba5447ae013#l52 | 17:17 |
| lkcl | you can see the definition for the pseudocode for the "add" instruction | 17:18 |
| lkcl | 52 * add RT,RA,RB | 17:18 |
| ghostmansd | 3 and 2 are register numbers? | 17:18 |
| lkcl | yes. | 17:18 |
| ghostmansd | and these map to RA and RB respectively? | 17:18 |
| lkcl | add 1, 3, 2 | 17:18 |
| lkcl | maps to RT RA and RB respectively | 17:18 |
| lkcl | so we *expect*; | 17:19 |
| lkcl | RT=1 | 17:19 |
| lkcl | RA=3 | 17:19 |
| lkcl | RB=2 | 17:19 |
| ghostmansd | got it | 17:19 |
| lkcl | so, when searching in the logs, first we searched "call add" | 17:19 |
| lkcl | then searched (from that point) for "reading read RA" | 17:19 |
| lkcl | we *expect* it to say, "reading reg RA 3" | 17:19 |
| lkcl | and, ta-daaa, that's exactly what it says | 17:20 |
| ghostmansd | aha, I see, and next it takes RB 2 | 17:20 |
| lkcl | yees | 17:20 |
| ghostmansd | what 0 stands for? | 17:20 |
| ghostmansd | reading reg RA 3 0 | 17:20 |
| lkcl | well, let's look in the source code back in caller.py | 17:20 |
| ghostmansd | initial value? | 17:21 |
| lkcl | log('reading reg %s %s' % (name, str(regnum)), is_vec) | 17:21 |
| ghostmansd | initial_regs = [0] * 32 | 17:21 |
| lkcl | decoder/isa/caller.py | 17:21 |
| lkcl | no: it's whether the register has been marked as "Vectorised" or not | 17:21 |
| lkcl | (we're extending the Power ISA to add Cray-style vectors) | 17:22 |
| ghostmansd | ah, OK | 17:22 |
| lkcl | so in this case, because it is a v3.0B 32-bit instruction, is_vec is expected to be zero | 17:22 |
| lkcl | we can do an SVP64 unit test later, at some point | 17:22 |
| lkcl | or you can try it yourself - test_caller_svp64.py | 17:22 |
| lkcl | ok so | 17:22 |
| lkcl | the next thing we should expect in the log is "writing 1 gpr" | 17:23 |
| lkcl | i really should invert that, make it easier to search for | 17:23 |
| lkcl | done :) | 17:24 |
| lkcl | if you "git pull" you should get that | 17:24 |
| lkcl | in caller.py you should see this: | 17:25 |
| lkcl | log('writing %s %s %s' % (ftype, regnum, str(output)), | 17:25 |
| lkcl | is_vec) | 17:25 |
| lkcl | and the corresponding log output is: | 17:25 |
| lkcl | writing gpr 1 SelectableInt(value=0x5555, bits=64) 0 | 17:26 |
| lkcl | GPR setitem 1 SelectableInt(value=0x5555, bits=64) | 17:26 |
| lkcl | with me so far? :) | 17:26 |
| ghostmansd | just a moment, I have to stash changes in test_caller.py | 17:26 |
| ghostmansd | before the pull | 17:26 |
| lkcl | ok. btw feel free to commit comments (meaningful ones!) and extra log() statements | 17:27 |
| ghostmansd | aaah, `git config` is needed | 17:27 |
| lkcl | but you know the rules about adding comments, don't add _too many_ comments :) | 17:27 |
| ghostmansd | I believe I did it before schroot | 17:27 |
| ghostmansd | the best comments are those which are not present, since the code speaks for itself :-) | 17:28 |
| lkcl | mmm clearly you've not read rocket-chip's source code :) | 17:28 |
| lkcl | only people with an IQ well north of 160 can understand it | 17:28 |
| ghostmansd | ah, the good ol' "you are not expected to understand this" from 6th edition | 17:29 |
| lkcl | the OO abstraction is clean and clear... and so deep in the number of levels it's almost impossible to comprehend. annoyingly, we risk the same thing with libre-soc's HDL | 17:29 |
| lkcl | one of the downsides of OO class inheritance | 17:30 |
| ghostmansd | ok, I believe we're synced now at f3c6dc5 | 17:30 |
| ghostmansd | let me unstash and re-run | 17:30 |
| lkcl | excellent. ok so all i did was a 1-line change swapping "writing 1 gpr" to "writing gpr 1" | 17:30 |
| lkcl | ok | 17:30 |
| ghostmansd | gpr stands for general-purpose register, right? | 17:31 |
| lkcl | yes. | 17:31 |
| lkcl | fpr Floating Point Register | 17:31 |
| lkcl | CR - Condition Register | 17:31 |
| ghostmansd | OK, I'm at writing `gpr 1 SelectableInt(value=0x5555, bits=64) 0` | 17:31 |
| lkcl | excellent! | 17:31 |
| lkcl | ok, now look at the unit test again | 17:32 |
| lkcl | initial_regs[3] = 0x1234 | 17:32 |
| lkcl | initial_regs[2] = 0x4321 | 17:32 |
| lkcl | and it's an "add" operation on 3 and 2 | 17:32 |
| lkcl | so | 17:32 |
| lkcl | duh | 17:32 |
| lkcl | we expect the answer 0x5555 | 17:32 |
| lkcl | which is why the unit test does | 17:32 |
| lkcl | self.assertEqual(sim.gpr(1), SelectableInt(0x5555, 64)) | 17:32 |
| lkcl | and, ta-daaaaa, it passes! | 17:32 |
| lkcl | now, to double-check, if you look _right_ at the end of the regfile, you see this: | 17:33 |
| lkcl | execute one, CIA NIA 4 8 | 17:33 |
| lkcl | reg 0 00000000 00005555 00004321 00001234 00000000 00000000 00000000 00000000 | 17:33 |
| lkcl | so again you can "triple-check" the contents of the "regfile" (simulated) | 17:33 |
| ghostmansd | "regfile" stands for log? | 17:34 |
| ghostmansd | execute one, CIA NIA 4 8 | 17:34 |
| ghostmansd | reg 0 00000000 00005555 00004321 00001234 00000000 00000000 00000000 00000000 | 17:34 |
| ghostmansd | reg 8 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 | 17:34 |
| ghostmansd | reg 16 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 | 17:34 |
| ghostmansd | reg 24 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 | 17:34 |
| ghostmansd | __eq__ SelectableInt(value=0x5555, bits=64) SelectableInt(value=0x5555, bits=64) | 17:34 |
| ghostmansd | eq 21845 21845 True | 17:34 |
| lkcl | regfile i mean "register file", in this case, the gpr. | 17:34 |
| lkcl | yes. | 17:34 |
| ghostmansd | aha, that's the gpr dump | 17:34 |
| lkcl | btw do be careful not to paste too many lines, IRC servers get unhappy | 17:34 |
| lkcl | yes | 17:34 |
| ghostmansd | sorry, I didn't know :-( | 17:35 |
| lkcl | so keep it to maximum... around.... 5 lines | 17:35 |
| lkcl | they rate-limit you | 17:35 |
| ghostmansd | quite a strict limitation! | 17:35 |
| lkcl | anything over that it's a good idea to use pastebin | 17:35 |
| lkcl | if you're handling 100,000 users (like libera is) it's a gooood idea :) | 17:35 |
| ghostmansd | I already got banned by gitolite, please, let IRC have mercy :-) | 17:35 |
| lkcl | haha | 17:36 |
| ghostmansd | OK, will use pastebin next time | 17:36 |
| lkcl | it's a tricky one, we're under "audit" conditions, and pastebin is transitory. sigh | 17:36 |
| lkcl | really we should run our own pastebin and make it permanent. but then (sigh) have to deal with spammers | 17:37 |
| lkcl | urrrr | 17:37 |
| lkcl | anyway | 17:37 |
| lkcl | so you see the chain connecting log to unit test to caller.py | 17:37 |
| lkcl | and to the MDWN file with the specification | 17:37 |
| lkcl | fixedarith.mdwn | 17:37 |
| ghostmansd | hm, I see only fixedarith.py in decoder/isa | 17:38 |
| lkcl | ah open a new terminal | 17:39 |
| lkcl | and go to the openpower-isa directory | 17:39 |
| ghostmansd | ah, I see | 17:39 |
| ghostmansd | ../../openpower/isa/fixedarith.mdwn | 17:39 |
| lkcl | lkcl@fizzy:~/src/libresoc/openpower-isa | 17:39 |
| lkcl | yes. | 17:39 |
| lkcl | excellent | 17:39 |
| lkcl | because it's, well, not-exactly-source (src) | 17:39 |
| lkcl | but... actually, it is :) | 17:39 |
| lkcl | so, good, you found fixedarith.py, that's excellent | 17:40 |
| lkcl | if you open up both files side-by-side that's a very useful exercise | 17:40 |
| lkcl | this is where it gets fascinating (for me) | 17:40 |
| lkcl | we did NOT write fixedarith.py by hand | 17:40 |
| lkcl | it was *COMPILED*... from fixedarith.mdwn | 17:41 |
| lkcl | (!!!!!) | 17:41 |
| ghostmansd | what??? | 17:41 |
| lkcl | yes. | 17:41 |
| lkcl | teehee | 17:41 |
| ghostmansd | I thought it's usually vice-versa :-D | 17:41 |
| lkcl | i actually took the OpenPOWER v3.0B ISA pseudocode | 17:41 |
| lkcl | which says, "warning this pseudocode is not supposed to be executable" | 17:41 |
| lkcl | and... well... made it executable | 17:41 |
| lkcl | or... language-translatable | 17:42 |
| lkcl | Boris Shingarov, of Labware, is REALLY happy with that | 17:42 |
| lkcl | because he is going to use these markdown files to create Formal Proofs of the Power ISA | 17:42 |
| lkcl | (!!!) | 17:42 |
| ghostmansd | :-D | 17:43 |
| lkcl | have you heard of python-ply at all? | 17:43 |
| ghostmansd | never | 17:43 |
| lkcl | https://www.dabeaz.com/ply/ | 17:43 |
| ghostmansd | ah, lex-yacc | 17:44 |
| lkcl | i love it. it's.. yes | 17:44 |
| ghostmansd | well, that one I used once | 17:44 |
| lkcl | so... next rabbithole? :) | 17:44 |
| ghostmansd | but not in python form for sure | 17:44 |
| lkcl | yeah. it's out of fashion because it's not as efficient as doing the parser manually yourself | 17:44 |
| ghostmansd | they give classes, lol | 17:44 |
| ghostmansd | https://www.dabeaz.com/compiler.html | 17:44 |
| lkcl | gcc dropped lex/yacc (flex / bison) some time ago | 17:44 |
| lkcl | david's great, i really like his work | 17:45 |
| ghostmansd | you mean, they switched to hand-written grammar? | 17:45 |
| ghostmansd | I bet C++ was the reason | 17:45 |
| lkcl | hand-written parsers | 17:45 |
| lkcl | they're faster than anything LALR parsers can achieve | 17:45 |
| ghostmansd | C++ is extremely awful in parseability anyway | 17:46 |
| lkcl | ok so if you open up decoder/pseudo/parser.py | 17:46 |
| lkcl | if you've ever looked at flex/bison stuff, you should easily and instantly recognise the BNF in the python comments | 17:46 |
| lkcl | this is how ply works: it uses the *comment* field to house the BNF syntax to create the LALR parser | 17:47 |
| lkcl | when i first encountered PLY like... 10+ years ago.. i loved this idea :) | 17:47 |
| lkcl | you'll particularly like this: i originally started the compiler from the GardenSnake.py python-ply example! | 17:48 |
| lkcl | so the grammar is *only slightly* python-like. the syntax (the symbols) are what have changed. | 17:48 |
| lkcl | "<-" for assignment operator | 17:48 |
| lkcl | instead of "=" | 17:48 |
| lkcl | but this is in the OpenPOWER v3.0B pseudo-code specification, so we can't change it | 17:49 |
| lkcl | or if we try, we meet with strong resistance | 17:49 |
| ghostmansd | would it help if you suggest ":="? :-) | 17:49 |
| lkcl | the OpenPOWER spec is now under the control of the OpenPOWER Foundation Working Group. changes are controlled by a "proper process" | 17:50 |
| lkcl | it's all posh and everything :) | 17:50 |
| lkcl | so anyway, if you scroll down to the bottom of parser.py, you can see it still says "GardenSnakeCompiler" :) | 17:51 |
| lkcl | so here's an experiment, to see how this fits together | 17:51 |
| ghostmansd | let's check | 17:52 |
| lkcl | in fixedarith.mdwn find this line: | 17:52 |
| lkcl | Pseudo-code: | 17:52 |
| lkcl | 17:52 | |
| lkcl | RT <- (RA) + (RB) | 17:52 |
| lkcl | (line 59) | 17:52 |
| lkcl | and change that "+" to a "-" | 17:53 |
| ghostmansd | done | 17:53 |
| lkcl | ok so in fixedarith.py, look for "def op_add(...) | 17:54 |
| lkcl | " | 17:54 |
| lkcl | and check it says RT = RA + RB | 17:54 |
| lkcl | return (RT,) | 17:54 |
| ghostmansd | aha | 17:54 |
| lkcl | which, we expected, given that the unmodified version is RT <- (RA) + (RB) | 17:54 |
| lkcl | so now run the following command: | 17:54 |
| lkcl | pywriter noall fixedarith | 17:55 |
| lkcl | or | 17:55 |
| lkcl | pywriter noall fixedarith > /tmp/verboseloadofcrud | 17:55 |
| lkcl | this takes a while (it has to compile about 100+ instructions) | 17:55 |
| lkcl | the "noall" btw is to stop pywriter.py from overwriting decoder/isa/all.py | 17:57 |
| lkcl | this should be a lot less klunky but hey | 17:57 |
| ghostmansd | ok, it's completed | 17:58 |
| lkcl | dang that was a while :) | 17:58 |
| lkcl | i forget how ridiculously fast my laptop is | 17:59 |
| lkcl | okay! | 17:59 |
| lkcl | so now if you inspect fixedarith.py again | 17:59 |
| lkcl | def op_add(self, RA, RB): | 17:59 |
| lkcl | RT = RA - RB | 17:59 |
| lkcl | ta-daaaaa! | 17:59 |
| ghostmansd | it might actually completed a bit ago, I was looking at parser.py :-) | 17:59 |
| lkcl | ah lol | 17:59 |
| lkcl | parser.py is _basically_ a subset of python... just with different symbols | 18:00 |
| ghostmansd | aha, yes, it's changed | 18:00 |
| lkcl | and, crucially, it's a single-pass compiler | 18:00 |
| lkcl | exccellent | 18:00 |
| lkcl | ok so now if we re-run the unit test | 18:00 |
| lkcl | test_caller.py | 18:00 |
| lkcl | it should FAIL | 18:00 |
| lkcl | AssertionError: SelectableInt(value=0xffffffffffffcf13, bits=64) != SelectableInt(value=0x5555, bits=64) | 18:00 |
| ghostmansd | AssertionError: SelectableInt(value=0xffffffffffffcf13, bits=64) != SelectableInt(value=0x5555, bits=64) | 18:01 |
| lkcl | ta-daaaa! | 18:01 |
| lkcl | best to revert that change to fixedarith.mdwn, you don't want _all_ unit tests failing that involve arithmetic :) | 18:01 |
| ghostmansd | let's recap, OK? | 18:01 |
| lkcl | sure | 18:02 |
| ghostmansd | we have markdown which is parsed by parser.py | 18:02 |
| lkcl | if you summarise, i can check | 18:02 |
| lkcl | yes. | 18:02 |
| ghostmansd | it, in turn, generates some python files, which generate the ISA based on contents of markdown | 18:02 |
| lkcl | (which is a python-based compiler with an altered set of tokens) | 18:03 |
| lkcl | yes | 18:03 |
| ghostmansd | I don't get, how does code from fixedarith.py makes its way into the ISA | 18:04 |
| lkcl | ok, that goes through decoder/isa/all.py | 18:04 |
| ghostmansd | I mean, where's the mapping between `def op_add(...)` and "add ..." | 18:04 |
| lkcl | which is also auto-generated | 18:04 |
| lkcl | so if you open decoder/isa/all.py as well | 18:04 |
| ghostmansd | ah, `self.instrs = `? | 18:05 |
| lkcl | you see it's blindingly obvious: it basically "imports" aaalll of fixedarith.py. ... blah blahh... | 18:05 |
| lkcl | yeeeess. | 18:05 |
| ghostmansd | ok, we have dict'o'dicts | 18:05 |
| lkcl | ah no, it's a dict *merged* from dicts | 18:05 |
| lkcl | that's the ** | 18:05 |
| ghostmansd | ah, sorry, I meant this | 18:05 |
| ghostmansd | yep | 18:05 |
| lkcl | ok | 18:05 |
| lkcl | so now if you look in caller.py | 18:05 |
| lkcl | or... sorry, all.py | 18:06 |
| lkcl | notice this: | 18:06 |
| ghostmansd | I have one more question | 18:06 |
| lkcl | class ISA(ISACaller, | 18:06 |
| lkcl | yes, sure | 18:06 |
| ghostmansd | I don't understand where `class fixedarith` is used | 18:07 |
| lkcl | ok, that's in all.py | 18:07 |
| ghostmansd | ah, OK, that's in inheritance | 18:08 |
| lkcl | class ISA(ISACaller, ..... --->fixedarith<----, ....) | 18:08 |
| lkcl | yes | 18:08 |
| ghostmansd | fixedarith_instrs['add'] = instruction_info(func=op_add, | 18:08 |
| lkcl | blah blah, yes | 18:08 |
| ghostmansd | OK, so we inherit class fixedarith and "add" gets associated with one of its methods | 18:08 |
| ghostmansd | right? | 18:08 |
| lkcl | so each fixedarith.py adds a factory dictionary of functions | 18:08 |
| lkcl | yeees. | 18:08 |
| lkcl | and theeeen | 18:08 |
| lkcl | back in caller.py def call(....) | 18:09 |
| lkcl | line 1139 | 18:09 |
| lkcl | info = self.instrs[name] | 18:09 |
| lkcl | ta-daaa | 18:09 |
| lkcl | we have the function associated with the instruction. | 18:09 |
| lkcl | performance here is *not* a high priority | 18:10 |
| ghostmansd | aha, got it | 18:10 |
| lkcl | the idea is to have "something that works" | 18:10 |
| lkcl | i get i think.... 2,000 instructions per second out of this simulator? | 18:10 |
| lkcl | and i'm astounded and delighted it's that fast | 18:10 |
| ghostmansd | where are the opcodes? | 18:10 |
| lkcl | ah, i'm so glad you asked :) | 18:11 |
| ghostmansd | I mean, I got the disasm part | 18:11 |
| lkcl | ready for the next rabbithole? :) | 18:11 |
| ghostmansd | but there for sure should be opcodes | 18:11 |
| ghostmansd | yes, sure | 18:11 |
| lkcl | yes. | 18:11 |
| lkcl | so there are two parts to this. | 18:11 |
| lkcl | first, if you grab the v3.0B PDF | 18:11 |
| lkcl | https://ftp.libre-soc.org/ | 18:11 |
| lkcl | https://ftp.libre-soc.org/PowerISA_public.v3.0B.pdf | 18:11 |
| ghostmansd | there's .C | 18:11 |
| ghostmansd | should I use .B? | 18:12 |
| lkcl | .B is the one that's "approved" by the OPF EULA | 18:12 |
| lkcl | but i think they updated to C | 18:12 |
| lkcl | let's go with C | 18:12 |
| lkcl | this is not a good idea to read all at once btw :) | 18:13 |
| lkcl | note page 6 section 1.3.4 contains the pseudo-code syntax? | 18:13 |
| ghostmansd | ok, I got it | 18:13 |
| lkcl | this is where we had to change the GardenSnake.py python symbols, "=", into "<-" etc. etc. | 18:14 |
| ghostmansd | aha, I see | 18:14 |
| lkcl | now... let's jump to.... 1.6 | 18:14 |
| lkcl | p12 | 18:14 |
| lkcl | so if you find XO-Form | 18:15 |
| lkcl | which is err.. 1.6.19, page 15 | 18:15 |
| ghostmansd | aha | 18:15 |
| ghostmansd | this is bitwise, right? | 18:15 |
| ghostmansd | is it little or big? | 18:15 |
| lkcl | now if you look back in fixedarith.mdwn, line 50? | 18:15 |
| lkcl | it's... MSB0 order | 18:16 |
| lkcl | bit/little endian is *separate* from MSB0 order numbering | 18:16 |
| lkcl | MSB0 ordering: the MOST significant bit is numbered ZERO | 18:16 |
| * lkcl sob | 18:16 | |
| ghostmansd | ok | 18:16 |
| lkcl | i spent 4 months getting used to that and i still get it wrong | 18:16 |
| lkcl | BE/LE is just that the instruction fetch automatically byte-reverses the data. | 18:17 |
| lkcl | Microwatt added BE/LE support in a patch that was only 80 lines! | 18:17 |
| lkcl | BE/LE is *selectable* at runtime with a bit in the Machine Status Register (MSR) | 18:18 |
| lkcl | but let's ignore that one for now :) | 18:18 |
| lkcl | so, section 1.6.19 you see the table? | 18:18 |
| lkcl | PO RT RA RB XO | 18:18 |
| ghostmansd | let's take 0xdeadbeef | 18:18 |
| lkcl | and some bit-fields OE and Rc (OE is "overflow enable", Rc is "Result condition") | 18:19 |
| ghostmansd | [1]1011110101011011011111011101111 | 18:19 |
| lkcl | so that is 32-bit | 18:19 |
| ghostmansd | the bit I selected is MSB0? | 18:19 |
| ghostmansd | so they move l2r? | 18:19 |
| lkcl | so... err.. you're asking someone who is slightly logic-dyslexic, but yes, i believe so. | 18:19 |
| ghostmansd | or right to left? | 18:19 |
| lkcl | the total opposite of what everyone else in computer science uses and expects, basically | 18:20 |
| lkcl | why the hell they chose this is beyond me | 18:20 |
| ghostmansd | :-D | 18:20 |
| ghostmansd | OK, left-to-right, right? | 18:20 |
| lkcl | i so did *NOT* want to deal with this that we created a class called SelectableInt | 18:20 |
| lkcl | left being numbered zero (because it's first, i assume that's the logic) | 18:21 |
| lkcl | SelectableInt is really important to the python-based simulator. decoder/selectable_int.py | 18:21 |
| ghostmansd | # NOTE: POWER 3.0B annotation order! see p4 1.3.2 | 18:22 |
| ghostmansd | # MSB is indexed **LOWEST** (sigh) | 18:22 |
| ghostmansd | OK, I see :-) | 18:22 |
| lkcl | after 6 months you get used to it, i'm told. | 18:22 |
| ghostmansd | lol | 18:22 |
| lkcl | i am still having kittens :) | 18:22 |
| lkcl | ok so where were we. | 18:22 |
| lkcl | yes. fixedarith,mdwn, line 50 | 18:22 |
| ghostmansd | at XO-FORM | 18:23 |
| lkcl | yes. | 18:23 |
| lkcl | so back to v3.0B PDF, XO-Form, ta-daaa, now we know what bit-fields to read for the opcode | 18:23 |
| lkcl | and you now jump tooOooo... 1sec... | 18:24 |
| lkcl | p67 | 18:24 |
| ghostmansd | OK, we basically have 4 options wrt OE and RC bits | 18:24 |
| lkcl | Section 3.3.9 v3.0B | 18:24 |
| lkcl | yes. | 18:24 |
| ghostmansd | these are enumerated at mdwn | 18:24 |
| ghostmansd | what / stands for? | 18:24 |
| ghostmansd | and ///? | 18:25 |
| lkcl | OE is "overflow enable" which affects XER register bits | 18:25 |
| ghostmansd | at XO form patterns | 18:25 |
| ghostmansd | no, I mean, the instruction pattern | 18:25 |
| lkcl | that means "there's some bits that are separated but ignored" | 18:25 |
| lkcl | so "/" means "there are 2 bits" | 18:25 |
| lkcl | "///" means "there are 4 bits something / {ignored} / {ignored} / {ignored} / something | 18:26 |
| ghostmansd | ehm, how could it be? shouldn't it be 1 and 3 respectively? | 18:26 |
| ghostmansd | I see / under bit 21 | 18:26 |
| lkcl | 1 sec let me find the page again | 18:26 |
| ghostmansd | that's the only one | 18:26 |
| ghostmansd | page 15 | 18:26 |
| ghostmansd | 1.6.19 | 18:26 |
| lkcl | ok yes, they missed out one "/" there from the spec :) | 18:27 |
| lkcl | ok so i was slightly wrong, "/" means "this bit is ignored" | 18:28 |
| lkcl | "///" means "all bits are ignored" | 18:28 |
| ghostmansd | so we have 4 types of instructions in XO-FORM, right? | 18:29 |
| lkcl | yes. | 18:29 |
| ghostmansd | two take into account OE and Rc | 18:29 |
| lkcl | luckily, all fields with the same name are all in the same position. | 18:29 |
| lkcl | idiotically, there are some fields in the same "Form" which are *not* the same. | 18:29 |
| lkcl | grrr | 18:29 |
| lkcl | ready for the next surprise? :) | 18:30 |
| ghostmansd | OK, let's move on to the next part. I don't quite yet understand the pattern matching, especially for how it maps to add[###] variants, but OK. | 18:30 |
| lkcl | https://git.libre-soc.org/?p=openpower-isa.git;a=blob;f=openpower/isatables/fields.text;hb=HEAD | 18:30 |
| lkcl | or, if you want to see the local copy, openpower-isa/openpower/isatables/fields.txt in your checked out copy | 18:31 |
| lkcl | look familiar? | 18:31 |
| ghostmansd | yep, that's a copy of pdf :-) | 18:31 |
| lkcl | the surprise: it's been made machine-readable | 18:31 |
| lkcl | https://git.libre-soc.org/?p=openpower-isa.git;a=blob;f=src/openpower/decoder/power_fields.py;hb=HEAD | 18:32 |
| ghostmansd | day of surprises, lol :-D | 18:32 |
| lkcl | :) | 18:32 |
| lkcl | you notice how the "|"s all line up in the fields.txt file? | 18:32 |
| ghostmansd | 161 "OE": self.FormXO.OE, | 18:33 |
| ghostmansd | 164 "CR": self.FormXL.XO, | 18:33 |
| lkcl | that's so that the power_fields.py can determine *exactly* where the bits start, by matching up the character count with the header at the top of each section | 18:33 |
| ghostmansd | yep, I see the magic :-) | 18:33 |
| lkcl | those are a bit of a cheat, they really shouldn't be there | 18:33 |
| lkcl | i copied those from Microwatt | 18:33 |
| lkcl | they're all unique (fortunately) | 18:33 |
| lkcl | caller.py does *not* use them, it *only* grabs the fields from the Form (self.FormXO) and drops them into the ISACaller function | 18:34 |
| lkcl | oh btw, one important thing: that's done with a very very special decorator, @inject | 18:34 |
| lkcl | at the end of caller.py | 18:35 |
| lkcl | i found that code-snippet on stackexchange | 18:35 |
| lkcl | it's a massive cheat | 18:35 |
| ghostmansd | just a moment, I need to take a deeper look at powerfields | 18:35 |
| lkcl | so that the pseudo-code doesn't have to be... ok :) | 18:35 |
| lkcl | i must apologise, power_fields.py is not properly commented | 18:36 |
| ghostmansd | there are some comments :-) | 18:36 |
| ghostmansd | 201 # if heading.startswith('1.6.28'): # skip instr fields for now | 18:36 |
| ghostmansd | 202 # break | 18:36 |
| ghostmansd | :-D | 18:36 |
| lkcl | mercifully it's short :) | 18:36 |
| ghostmansd | let's simply check how I suppose it works... | 18:37 |
| lkcl | line 254: you can actually run it | 18:37 |
| lkcl | and it will dump what it parses | 18:37 |
| ghostmansd | ah, ok, let's go this way | 18:37 |
| lkcl | then you can compare what it produces against the original, fields.txt | 18:38 |
| lkcl | most of these we do not - and will not - actually use | 18:38 |
| lkcl | but note that there are *two* phases. | 18:38 |
| ghostmansd | hm | 18:38 |
| lkcl | Phase 1: parse everything befoer 1.6.28 | 18:38 |
| ghostmansd | I'm not sure how it works | 18:39 |
| lkcl | Phase 2: parse section 1.6.28 | 18:39 |
| ghostmansd | you mean 1.6.16? | 18:39 |
| lkcl | no, 1.6.28 | 18:39 |
| ghostmansd | we're looking at XO-FORM I thought | 18:39 |
| ghostmansd | # 1.6.16 XO-FORM | 18:39 |
| lkcl | oh right, ok. | 18:39 |
| lkcl | right, i was talking about power_fields.py in general. | 18:40 |
| lkcl | it does two phases. | 18:40 |
| lkcl | Phase 1: read sections with headers 1.6.1....27 | 18:40 |
| lkcl | Phase 2: read *only* section "1.6.28 Instruction Fields" | 18:40 |
| ghostmansd | Ah, OK | 18:40 |
| lkcl | basically, they are *exactly* the same information | 18:41 |
| lkcl | just expressed in 2 different ways. | 18:41 |
| lkcl | i parse them both in order to cross-check each other | 18:41 |
| ghostmansd | I don't get why we had OE (21) | 18:41 |
| ghostmansd | but in logs we have " field OE BitRange([(0, 21)])" | 18:41 |
| ghostmansd | shouldn't it be (21, 21)? | 18:42 |
| lkcl | errr... i don't know either :) | 18:42 |
| lkcl | ah! because it's a map | 18:42 |
| lkcl | Bit 0 of OE is *MAPPED* to Bit 21 of the opcode | 18:43 |
| ghostmansd | ah, ok, got it | 18:43 |
| lkcl | and because it's an OrderedDict, the "print" statement prints out tuples of (key, value), (key, value) | 18:43 |
| ghostmansd | I'd have expected a simple thing like start-end | 18:43 |
| ghostmansd | but OK | 18:44 |
| lkcl | because... 1 sec.... | 18:44 |
| lkcl | there is a reason... | 18:44 |
| ghostmansd | or [21] | 18:44 |
| lkcl | ok found one | 18:44 |
| lkcl | look at v3.0C p29 | 18:44 |
| lkcl | Formats: X | 18:44 |
| lkcl | SX,S (28,6:10) | 18:44 |
| lkcl | Fields SX | 18:44 |
| lkcl | no hang on... a better one... | 18:45 |
| lkcl | p21 | 18:45 |
| lkcl | XO (21:24,26:28) | 18:45 |
| lkcl | that says: | 18:45 |
| lkcl | XO is (24-21+1 + 28-26-1) bits | 18:45 |
| lkcl | which is | 18:45 |
| ghostmansd | I must be missing the place | 18:46 |
| lkcl | 4+3=7 | 18:46 |
| lkcl | 7 bits | 18:46 |
| lkcl | v3.0C p21 | 18:46 |
| ghostmansd | I'm at chapter 2 branch facility | 18:46 |
| ghostmansd | that's p29 | 18:46 |
| lkcl | Chapter 1 Introduction | 18:46 |
| lkcl | p21 | 18:46 |
| lkcl | or if you prefer, line 902 of fields.txt | 18:47 |
| lkcl | Formats: XX2, XO (21:24,26:28) | 18:47 |
| lkcl | this says: | 18:47 |
| lkcl | "XO is 7 bits. bit 0 of XO is bit 21. | 18:48 |
| lkcl | bit 1 of XO is bit 22 in the opcode | 18:48 |
| lkcl | bit 2 of XO is bit 23 in the opcode | 18:48 |
| lkcl | bit 3 of XO is bit 24 in the opcode | 18:48 |
| lkcl | bit 4 of XO is bit *26* of the opcode | 18:48 |
| lkcl | bit 5 of XO is bit *27* of the opcode | 18:48 |
| ghostmansd | ah, I see, it's split | 18:48 |
| lkcl | bit 6 of XO is bit *28* .... | 18:49 |
| lkcl | yeeeees. | 18:49 |
| lkcl | hence, you cannot just have a list. | 18:49 |
| lkcl | annoying, isn't it? | 18:49 |
| ghostmansd | so it uses 21:24, and 26:28 | 18:49 |
| lkcl | yeeees. | 18:49 |
| ghostmansd | yep, but still beter than x86 | 18:49 |
| lkcl | lol | 18:49 |
| ghostmansd | at least the encoding format can be comprehended by mortals | 18:49 |
| lkcl | bear in mind, those are MSB0 numbers! | 18:49 |
| ghostmansd | well, nothing's perfect | 18:49 |
| ghostmansd | :-D | 18:49 |
| lkcl | so it's actually, in "normal mortals" language: | 18:49 |
| lkcl | bit 0 is (31-21) of the opcode | 18:50 |
| lkcl | ... | 18:50 |
| lkcl | ... | 18:50 |
| lkcl | bit 6 of XO is bit (31-28) | 18:50 |
| ghostmansd | no wonder you have a specialized class for this, lol | 18:50 |
| lkcl | hence, again, why SelectableInt exists | 18:50 |
| lkcl | yes :) | 18:50 |
| ghostmansd | OK, one mystery is solved | 18:51 |
| ghostmansd | now I get why we have a map | 18:51 |
| lkcl | so, your mission, should you choose to accept it, is over the next few months to create a version of parser.py that outputs *c* code instead of *python* code. | 18:51 |
| lkcl | fixedarith.c rather than fixedarith.py | 18:51 |
| ghostmansd | hm, but, still, one question... | 18:51 |
| lkcl | yah. there is one more part of the puzzle btw :) | 18:52 |
| lkcl | go for it | 18:52 |
| ghostmansd | 21:24,26:28 can be mapped as [[0,21], [1,22], ...] | 18:52 |
| lkcl | yes | 18:52 |
| ghostmansd | but cannot it be mapped as [21, 22, ...]? | 18:52 |
| ghostmansd | and len() would give the total number of bits | 18:52 |
| lkcl | it could, but for whatever reason i chose an OrderedDict. | 18:52 |
| ghostmansd | OK | 18:53 |
| lkcl | len(dict) also returns the number of items. | 18:53 |
| ghostmansd | that's not a critics, I'm just trying to get how it works :-) | 18:53 |
| lkcl | i think probably for speed, given that python list accesses are an O(N) lookup | 18:53 |
| lkcl | but dicts are O(1) | 18:53 |
| lkcl | given that accessing even just simple fields requires *bit-level* reconstruction of the actual number. | 18:54 |
| lkcl | *gibber* | 18:54 |
| lkcl | btw i have to get up and walk about, soon. been sitting too long. | 18:54 |
| ghostmansd | OK, let's continue next days, then | 18:55 |
| lkcl | i leave you with this: https://git.libre-soc.org/?p=openpower-isa.git;a=blob;f=openpower/isatables/minor_31.csv;hb=HEAD | 18:55 |
| ghostmansd | not the best goodbye I saw, BTW! | 18:55 |
| lkcl | which, if you check power_decoder.py (not too deeply, it's pretty intense) | 18:56 |
| lkcl | lol | 18:56 |
| lkcl | that's how we *actually* decode the opcode. | 18:56 |
| lkcl | everything and i mean everything is machine-readable files | 18:56 |
| lkcl | the entire spec | 18:56 |
| ghostmansd | frankly? it's damn good idea! | 18:57 |
| lkcl | ok now i am at least standing up, this is progress | 18:57 |
| ghostmansd | I like it | 18:57 |
| lkcl | i am amazed it's not more commonly done | 18:57 |
| ghostmansd | thank you for this intro! | 18:57 |
| lkcl | no problem :) | 18:57 |
| ghostmansd | sorry it took that much time | 18:57 |
| lkcl | no it's fine, it's really useful, someone else can look at this irc log now | 18:57 |
| lkcl | talk soon :) | 18:57 |
| ghostmansd | I think in the next few days I'll have to work at primary work mostly, but anyway, I'll ping here in IRC when I have some time | 18:58 |
| ghostmansd | only one question before you leave | 18:58 |
| ghostmansd | what _31 stands for before .csv? | 18:59 |
| openpowerbot | [irc] <programmerjake> btw, this room on libera.chat has had the matrix bridge blocking messages for the last 5 or so hours -- maybe we should move to the oftc room? | 19:53 |
| programmerjake | copying so hopefully people on matrix see: btw, this room on libera.chat has had the matrix bridge blocking messages for the last 5 or so hours -- maybe we should move to the oftc room? | 20:03 |
| openpowerbot | [irc] <programmerjake> https://matrix.to/#/#_oftc_#libre-soc:matrix.org | 20:08 |
| lkcl | ghostmansd: see power_decoder.py - minor_31.csv is the Primary Opcode 31 | 21:01 |
| lkcl | all instructions with bits 0..5 equal to 31 are in minor.31.csv | 21:02 |
| lkcl | programmerjake: openpowerbot bridges between oftc and libera for us, thanks to toshaan | 21:02 |
| lkcl | ghostmansd: you can see the full opcode map starting at p1162 of v3.0C with Table 10 ("Primary Opcode Map") | 21:03 |
| openpowerbot | [irc] <programmerjake> yup, thx toshywoshy! | 21:04 |
| lkcl | and you'll see the green box "EXT31" which is minor_31.csv | 21:04 |
| lkcl | then, jump to p1176, Table 22 "EXT31" and you should see the operations there match up. | 21:04 |
| lkcl | example, cmpi is 00001 0000 | 21:04 |
| lkcl | cmpl sorry, screen's too small :) | 21:05 |
| lkcl | and, surpriise, line 18 in minor_31.csv is "0b0000100000,ALU,OP_CMP,RA,RB,...." | 21:06 |
| programmerjake | weekly meeting in a few min | 22:49 |
| lkcl | lxo rsc cesar klysm richardwilbur[m] ^ | 22:54 |
Generated by irclog2html.py 2.17.1 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!