Monday, 2021-08-16

programmerjakeidk where you got the idea that I just now suggested SV's semantics should change...I didn't (though I have and still do suggest they change for other unrelated reasons...)05:27
programmerjakeI was suggesting the simulator's design change...05:28
ghostmansdlkcl: nice patches, that's how I intended to do it today :-)07:51
ghostmansdYep, basically the whole thing was "create a new instance to pass a new value". I actually thought of using copy.deepcopy, though, because I didn't want to touch selectable int internals.07:56
ghostmansdI'll try to take a look at addg6s test today. What'd be the right way to check qemu's behavior? Is there the right way to launch our qemu? Should I simply cross-compile some addg6s code and run the binary on qemu? What are the BKMs?08:01
lkclprogrammerjake: ahh, right.  the simulator respects SVP64 Program Order.11:12
lkclanything that doesn't match that is a bug11:12
lkclghostmansd, python3 decoder/simulator/test_sim.py11:13
lkclprogrammerjake, you said, "<programmerjake> afaik >>>SimpleV<<<< is the only ISA..."11:15
lkclwhere retrospectively you probably meantto say "afaik ISACaller is the only simulator..."11:15
lkclunfortunately - annoyingly - we need to be able to cope with the situation where the pseudocode requires direct access to GPR and FPR, making modifications through a direct call to set GPR and FPR contents11:17
lkcli'm not going to make ISACaller do an entire copy of two 128-entry register files without a really *really* good reason11:18
lkclghostmansd: the one thing to watch out for is that element-width overrides are going to be a bit of a pig12:30
lkclit will no longer be "64 bit add", it will be "if elwidth==8 do 8-bit add elif elwidth==16 do 16-bit add ..."12:31
lkclaccessing the regfile as if it was a c union of {uint8_t *b; uint16_t *h; uint32_t *i; uint64_t *l}12:32
lkclhttps://libre-soc.org/openpower/sv/overview/#elwidths12:33
lkcltherefore, exactly at the point where the copy has been made, it will NOT be like that AT ALL12:34
lkclit will instead be something like:12:34
lkclreg_val = SelectableInt(self.gpr(regnum, srcstep, element_width))12:34
lkclwhere inside the GPR class, it will use the python equivalent of that typedef union12:35
lkclregnum will *NOT* be calculated as "RA + srcstep"12:36
lkclbecause that assumes that the element widths are fixed at 64-bit (8 bytes)12:36
lkclwhere (normally, reasonably) you *assume* that the regfile is organised into 64-bit quantities12:37
lkclelwidth overrides blow that out the water12:37
lkclprogrammerjake: if there are design ideas that have not already been discussed and evaluated, now is the time to raise them.12:39
lkclif they have already been discussed and evaluated, and you do not agree with the assessment that they not be implemented, please take the time to understand why the assessment was that they not be implemented12:41
lkclparticularly in light of the hard requirement that multi-issue execution not be impeded or compromised by any design decisions.12:42
lkclsome of the ideas that you, lxo, *and i* came up with all severely compromise multi-issue parallelism, creating dependency hazards either at decode or issue time.12:43
lkclsrcstep merged with SVP64 Branch-Conditional was the latest mistake that had to be backed out.12:44
lkclthis is how it is.12:44
lkclyou or i might *want* certain design features: reality is, they *have* to be practical to implement as multi-issue hardware12:45
lkclkey word: *practical*12:45
programmerjakewell, the idea of having operations' endian match memory endian (which greatly simplifies the compiler and allows big-endian SV code to run at the same speed as little-endian SV code since it doesn't need extra endian-swapping instructions) is practical to implement (just 1 2-in mux on register read/write) and matches existing precedent of VMX, that was shot down...17:30
lkclit's simple at the compiler, but absolutely horrendously complex at the register file and in the rest of the ISA.  plus, i can't cope with it.17:40
lkcli added a bit to REMAP which allows byte-swapping on a per-element basis17:40
lkclthat's as much as i can cope with.17:41
lkclnow, if someone (you) who advocates this idea *actually does the work* (rather than expecting me to implement it), creates a patch which adds it to both the HDL and the simulator then GREAT17:42
lkcland edits and updates the spec, and writes the unit tests, and everything else that's required17:42
lkcland demonstrates unequivocably that it's not going to cause massive headaches either in hardware or in actual understanding of the SVP64 concept, then it can be considered17:43
lkclif that's sounding like it's a hell of a lot of work, that's because it is17:43
lkclgiven that it's such an intrusive high-impact fundamental change, it needs to be properly thought through17:44
programmerjakeso, in other words, your fine with the idea if you don't have to implement it?17:45
lkclbecause i can't cope with it, mentally, given the logic-dyslexia and the complexity of everything else, basically, yes.17:45
programmerjakesounds good to me! I'd guess lxo will agree too...17:46
lkclclarification: i'm not "totally fine with it" until it's actually *proven* and demonstrated not to be so massively intrusive and complex that it damages our chances of adoption and completion of the milestones set to date17:46
lkclor results in unintended consequences well beyond "it's just a simple swap at the regfile"17:47
programmerjakeok17:48
lkclbefore getting to implementation however, a comparative analysis will be needed vs the current option: existing LD/ST-byte-reversed (ldbr) when Vectorised, plus REMAP with byte-reverse enabled.17:52
lkclwhat algorithms cannot be covered by that, where are they used, how often are they used, what is the cost17:53
lkcletc. etc. etc. etc.17:53
lkclall of which takes up a huge amount of time17:53
programmerjakeok...I'd expect the implementation to be simple enough that it might be easier to test it out by implementing it on a git branch or something...17:58
programmerjakethen we can play with algorithms on the actual simulator17:59
richardwilbur[m]I think there is benefit to testing on the simulator that is probably directly proportional to the quality of the simulation.  (It can be a pretty powerful test.)18:04
programmerjakedefinitely agree!18:10
Madan_KartheessaI have sent you  a message19:13
*** Adithya46 <Adithya46!~Adithya@2405:201:e004:386e:9599:7651:a545:ee6d> has left #libre-soc19:16
*** Adithya51 <Adithya51!~Adithya@2405:201:e004:386e:9599:7651:a545:ee6d> has left #libre-soc19:17
programmerjakehttps://xkcd.com/2503/19:37
ARJUN_NAGHello everyone20:01
programmerjakehi20:01
richardwilbur[m]Greetings Arjun.20:02
ARJUN_NAGHello richard20:02
ARJUN_NAGTq all20:02
ARJUN_NAGThanks for the warm welcome20:02
richardwilbur[m]@Madan:  Welcome to libre-soc IRC.20:03
lkclnice to have you on board, Arjun and Madan_Kartheessa20:12
lkclArjun i recommend you use a username that does not have a space in it20:13
lkcluse an underscore instead, arjun_nag20:13
programmerjakelkcl, the username has an underscore20:13
lkcldoh, that'll be hexchat removing them, sigh20:13
programmerjakeat least on my end20:14
lkclhttps://github.com/hexchat/hexchat/issues/244920:14
lkclahh20:14
lkclthat's annoying: it's an issue with libpango20:15
ghostmansd-pctest_sim.py doesn't seem to work on my side; `TypeError: __init__() got an unexpected keyword argument 'gdb_path'`20:15
ghostmansd-pcit looks like the arguments are wrong, the only argument is command, acting as list of strings20:16
lkclghostmansd-pc: 1 sec20:16
lkcli haven't run test_sim.py for a few months20:17
ghostmansd-pctried using `['powerpc64-linux-gnu-gdb'] + pygdbmi.DEFAULT_GDB_LAUNCH_COMMAND[1:]` instead20:17
ghostmansd-pcgot qemu-system-ppc64le: -machine powernv9: unsupported machine type20:17
richardwilbur[m]I also see the underscore in @ARJUN_NAG in Element.20:18
lkclghostmansd-pc: ok test_sim.py worked fine here20:18
lkcllet me check the code20:18
ghostmansd-pcand, in the end, 48 skips and 7 failures after that: `pygdbmi.constants.GdbTimeoutError: Did not get response from gdb after 1 seconds`20:18
ghostmansd-pccan it happen that we have different pygdbmi?20:19
programmerjakefor the unsupported machine type, that sounds like you have an old version of qemu...20:19
lkclhonestly it's not very stable: do "ps aux | grep qemu" and if it's still running, you have to *manually* kill it20:19
lkclii  qemu-system-ppc                                  1:4.2-3                                 amd64        QEMU full system emulation binaries (ppc)20:19
lkclr$ qemu-system-ppc64 --version20:20
lkclQEMU emulator version 4.2.0 (Debian 1:4.2-3)20:20
ghostmansd-pcno running qemu20:21
ghostmansd-pclet me re-check for version20:21
ghostmansd-pcyikes20:21
ghostmansd-pcextremely old...20:21
ghostmansd-pcQEMU emulator version 3.1.0 (Debian 1:3.1+dfsg-8+deb10u8)20:21
ghostmansd-pchm, this is strange, I recall I followed the routine at HDL workflow20:22
ghostmansd-pclet me re-check the script I used20:22
ghostmansd-pcit's simply `qemu-system-ppc`20:23
lkclwarl, they'rs yur prabbhlurm... :)20:23
ghostmansd-pc:-D20:23
ghostmansd-pcperhaps compiling from sources?20:23
lkclcan't find the appropriate mythbusters youtube video but hey20:23
ghostmansd-pcwould that work? not really HDL-ish, but hey20:24
lkcllet me check what's in debian/10 (which version of qemu)20:24
ghostmansd-pcHDL-workflowish, so to speak20:24
programmerjakeapparently powernv9 was added in qemu 4.2: https://wiki.qemu.org/ChangeLog/4.220:24
lkclhttps://packages.debian.org/search?keywords=qemu-system-ppc20:24
lkcldrat.20:24
lkcli installed qemu from debian/testing about 18 months ago20:25
lkclprogrammerjake: yeah that looks about right, 4.2 would have been in debian/testing back then20:25
ghostmansd-pc$ cat /etc/*release* | head -n120:25
ghostmansd-pcPRETTY_NAME="Debian GNU/Linux 10 (buster)"20:25
lkclthey've since moved on to... 5.2 https://packages.debian.org/bullseye/qemu-system-ppc20:26
lkclahh there's a debian/10 backport20:26
lkclurrr..20:26
lkclghostmansd-pc: try the debian/10 backport version20:26
ghostmansd-pchttps://packages.debian.org/buster-backports/qemu-system-ppc?20:27
lkclyes20:27
lkclyou know how to add backports to debian sources.list?20:27
richardwilbur[m]Does that need an edit to sources.list?20:27
lkcl(add it to /etc/apt/sources.list.d/buster_backports)20:27
ghostmansd-pcthat was lot ago20:28
lkclrichardwilbur[m], an entry in sources.list.d is better / cleaner20:28
ghostmansd-pcok, sounds fair :-)20:28
richardwilbur[m]Sorry, old debian-fu.20:28
ghostmansd-pcdeb http://deb.debian.org/debian buster-backports main20:29
ghostmansd-pcpoor man's jump into the future20:29
lkclngggh there's something else... have to do "apt-get install qemu-system-ppc/{insertversion}"20:30
ghostmansd-pcyep20:30
ghostmansd-pcapt-get install qemu-system-ppc/buster-backports20:30
lkclotherwise you have to piss about with /etc/apt/preferences20:30
lkclahhh awesome20:30
ghostmansd-pclocked and loaded!20:31
ghostmansd-pc$ qemu-system-ppc64 --version20:31
ghostmansd-pcQEMU emulator version 5.2.0 (Debian 1:5.2+dfsg-9~bpo10+1)20:31
ghostmansd-pcCopyright (c) 2003-2020 Fabrice Bellard and the QEMU Project developers20:31
ghostmansd-pcawesome, thanks, folks!20:31
richardwilbur[m]Probably better to use backports because you can pick and choose what you want to update rather than getting dragged kicking and screaming into new versions of pretty much everything.20:32
lkclhas to have a .list on the file extension20:32
ghostmansd-pcI wonder if I have pygdbmi OK20:33
lkclok err it's removing openbios-ppc but hey20:33
ghostmansd-pc  File "/usr/local/lib/python3.7/dist-packages/pygdbmi-0.10.0.1-py3.7.egg/pygdbmi/IoManager.py", line 98, in get_gdb_response20:34
ghostmansd-pc    "Did not get response from gdb after %s seconds" % timeout_sec20:34
ghostmansd-pcpygdbmi.constants.GdbTimeoutError: Did not get response from gdb after 1 seconds20:34
lkclhttps://git.libre-soc.org/?p=dev-env-setup.git;a=blob;f=install-hdl-apt-reqs;hb=HEAD20:34
lkclghostmansd-pc, you have powerpc64-linux-gnu-gdb installed?20:34
lkcl(that one has to be from source)20:34
ghostmansd-pcthis one is compiled iirc20:35
ghostmansd-pcyep, $ powerpc64-linux-gnu-gdb --version | head -n120:35
ghostmansd-pcGNU gdb (GDB) 8.320:35
lkcl$ powerpc64-linux-gnu-gdb --version20:35
lkclGNU gdb (GDB) 9.120:35
lkclno idea if that makes any difference20:35
ghostmansd-pcbut let me re-install hdl-apt-reqs with backports20:35
lkclok20:35
ghostmansd-pc(I guess it's there since now, right?)20:35
ghostmansd-pcin URL to git you posted20:35
lkclthen try just running openpower/simulator/qemu.py20:35
lkclbut do "make" first in the... errr.... which directory...20:36
ghostmansd-pcopenpower-isa20:36
lkclsimulator/qemu_test directory20:36
lkclopenpower-isa/src/openpower/simulator/qemu.py20:36
ghostmansd-pchm20:36
ghostmansd-pcOK, will be back soon20:36
lkcland openpower-isa/src/openpower/qemu_test/Makefile20:36
lkclhttps://git.libre-soc.org/?p=openpower-isa.git;a=blob;f=src/openpower/simulator/qemu_test/README.md;hb=HEAD20:37
lkcleliminate things from the chain to triage the problem20:38
ghostmansd-pcok, install-hdl-apt-reqs doesn't do anything, I'm already brand new as ever possible20:39
lkclexcellent20:39
lkclthat README will allow you to run qemu-system-ppc "directly" (not involving pygdbmi)20:40
lkclhowever it's a foreground process that eats console20:40
lkclso you need to run it in a separate xterm20:40
ghostmansd-pclaunch.sh hangs...20:41
ghostmansd-pcso is `qemu-system-ppc64 -machine powernv9 -nographic -s -S -kernel kernel.bin`20:41
lkclyes, that's the correct thing20:42
lkclthen in a *second* xterm you run the other command20:42
lkcl(see the README)20:42
ghostmansd-pcah, OK20:43
lkcli did say: it eats console input.  this is expected behaviour20:43
ghostmansd-pcthe readme doesn't mention that these are different consoles20:43
ghostmansd-pcit shows the commands as sequential20:43
ghostmansd-pcOK, I'm at 0x1020:44
ghostmansd-pcafter ENTER the breakpoint is added, and I reach it20:45
lkclexcellent.20:45
ghostmansd-pcI'm at kernel entry point20:45
ghostmansd-pc[    7.204320677,5] INIT: Starting kernel at 0x20000000, fdt at 0x306c8ef8 23888 bytes20:45
lkclok so both qemu-system-ppc and powerpc64-linux-gnu-gdb are "functional"20:45
ghostmansd-pcbrk is also at x2000000020:45
lkclper se20:45
lkclall good.20:45
ghostmansd-pcok, `kill` in gdb?20:45
lkclso that's not the source of the problem20:45
lkclprobably :)20:46
lkcli just killed it manually (killall qemu-system-ppc)20:46
ghostmansd-pcqemu-system-ppc64le: cannot set up guest memory 'pnv.ram': Cannot allocate memory20:47
lkclnext thing: do the programs... ahhh20:47
lkclthat's weird20:47
ghostmansd-pcI think this might have to do with it20:47
ghostmansd-pcincantation I use is `python3 src/openpower/simulator/test_sim.py`20:48
lkclhow much RAM do you have on your laptop?20:48
lkcl                  '-s', '-S', '-m', 'size=4096']20:48
lkcllaunch_args_le = ['qemu-system-ppc64le',20:48
lkclthat's asking for 4 GIGABYTES which is quite a lot20:48
lkclif physical RAM is the problem you might be able to bring that down to a much more sane size=1024 or even size=51220:49
ghostmansd-pclol, much less20:49
ghostmansd-pclet me first allocate more RAM to VM20:49
lkclwhat the heck we're doing asking for 4 gigabytes of RAM to execute programs of size 12 bytes is anybody's guess20:49
programmerjake^ probably part of why it's slow20:50
ghostmansd-pclmao, 640K ought to be enough for anybody20:50
programmerjakewell, if we're just testing a few instructions, try setting it to 1MB20:50
lkclpygdbmi is unfortunately rather badly designed20:51
ghostmansd-pc`qemu-system-ppc64le: warning: skiboot may not work with < 1GB of RAM`20:51
ghostmansd-pcthat's just a warning20:51
lkclit was never intended for mass-management to the extent we're using it20:52
lkclit has a timeout of 1 second on the socket20:52
ghostmansd-pcused `size=32`, perhaps way too little20:52
ghostmansd-pcsigh20:52
ghostmansd-pcstill `pygdbmi.constants.GdbTimeoutError: Did not get response from gdb after 1 seconds`20:52
lkclhmm20:53
lkclohhh hang on20:53
lkclyou're running qemu *inside* a VM, right?20:53
lkclan emulator inside an emulator, right?20:53
ghostmansd-pcyes, but this should work, isn't it?20:53
lkclyyeahhh but only if you enable the right options in the top-level emulator20:53
ghostmansd-pcI mean, nesting is usually supported20:53
programmerjakeor is the VM using native virtualization?20:53
lkcltcktcktck....20:54
ghostmansd-pclol, qemu is still running, even though the test's completed with fails20:54
* lkcl thinks20:54
ghostmansd-pcand it eats my CPU20:54
lkclah that's actually "bad"20:54
lkclkill it20:54
ghostmansd-pcyep20:54
ghostmansd-pcat least the laptop is warm, lol20:54
lkclha, it could just be that by running qemu-within-qemu it's just far too slow20:54
lkcl:)20:54
lkcland the 1 second timeout is simply too short20:55
programmerjakedo note that powernv9 means Power9 without virtualization20:55
lkclerrrmmmermermerm... https://github.com/cs01/pygdbmi/blob/master/pygdbmi/gdbcontroller.py20:56
lkclhttps://github.com/cs01/pygdbmi/blob/master/pygdbmi/constants.py20:56
lkclget_gdb_response is where the timeout is set to 1s...20:57
ghostmansd-pcconstants.py:3:DEFAULT_GDB_TIMEOUT_SEC = 120:58
lkclahh and also to the "write" function20:58
lkclok20:58
lkclyes.20:58
lkclso...20:58
ghostmansd-pcdo you think I should increase it?20:58
lkclself.gdb.write("data")....20:58
lkclyes.20:58
lkclyou can monkey-patch it20:58
lkclat the *very first* line20:59
lkclfrom pygdbmi import constants20:59
lkclthen20:59
ghostmansd-pcyou mean overriding it?20:59
lkclline 2: constants.DEFAULT_GDB_TIMEOUT_SEC = 520:59
ghostmansd-pcpygdbmi.DEFAULT_GDB_TIMEOUT_SEC = 666?20:59
lkclyes.20:59
ghostmansd-pcyep20:59
ghostmansd-pcwill do20:59
lkclmonkey-patching is such a dreadful hack...21:00
lkclonly really good reason i've seen it used is stackless21:00
lkclthey (hilariously) monkey-patch all python standard socket routines (!)21:01
ghostmansd-pcwhat for? they wanted some stuff before/after actual code?21:01
ghostmansd-pcno luck with changed constants, but at least laptop got warmer again :-)21:02
ghostmansd-pckilled the qemu beast again21:03
lkcl:)21:03
programmerjakeis the VM running out of ram and swapping like crazy!21:04
programmerjake?21:04
lkclif this is getting in the way, we can do it differently: hand-crafted values of addg6s.21:04
lkclcalculate what they're supposed to be, and add them to the test_caller_bcd.py21:05
lkclbut also add a qemu test as well, which one of us can run21:05
ghostmansd-pcprogrammerjake: 100% cpu at top21:05
ghostmansd-pcfwiw, maybe I could edit kernel.bin?21:05
* lkcl need to get up and walk about21:06
lkclkernel.bin should be a few bytes. it won't be the problem, itself21:06
ghostmansd-pcah, nice, there's even test.s which is used as source for kernel21:07
ghostmansd-pclkcl: nope, I mean, I could insert an instruction there and inspect the registers21:07
ghostmansd-pc(if $(AS) knows it)21:08
ghostmansd-pc(even if not, I could perhaps insert the instruction via objcopy or by other means)21:08
lkclghostmansd-pc, yes21:09
ghostmansd-pcaddg6s 1,1,121:09
ghostmansd-pcI did it for lulz, and it compiles21:09
lkclw00t21:10
ghostmansd-pcwe're not looking for an easy ways, right? ;-)21:10
ghostmansd-pc*way21:10
lkclomg we've regressed to l33t speek21:10
ghostmansd-pcthat's the direct consequence of qemu eating the whole CPU21:11
lkcljust the easiest repeatable way to confirm that the pseudocode's actually correct21:11
ghostmansd-pcok, I think I'll simply edit test.s, compile it and run with qemu, attaching with gdb21:11
ghostmansd-pcmaybe even some custom gdb script, don't know for sure yet21:12
lkclit's ISACaller that needs a repeatable test21:13
lkclalthough... ohh i get it21:13
lkclyou can hand-edit test.s21:13
lkclto generate arbitrary values21:13
lkclsee what those look like and insert them into test_caller_bcd.py21:13
lkclblech :)21:13
ghostmansdYep, that's what I wanna do :-)21:21
ghostmansdugly yet effective21:21
ghostmansdI even think... Hell, yeah, I can generate the whole listing via Python21:21
ghostmansd(not all values though)21:22
ghostmansdI'll have to re-check the instruction itself21:22
ghostmansdPerhaps at 20th read I'll finally understand why they need it, lol21:22
ghostmansdWhat I like in programming is that sometimes it naturally _forces_ you to be creative21:23
programmerjakelkcl, ghostmansd afaict qemu doesn't support any of cbcdtd, cdtbcd, or addg6s, I couldn't find them via grepping through qemu master or by going through all occurrences of "bcd"21:27
programmerjakehttps://lists.nongnu.org/archive/cgi-bin/namazu.cgi?query=cbcdtd&submit=Search%21&idxname=qemu-ppc&max=20&result=normal&sort=score21:29
programmerjakenot on mailing list either21:30
ghostmansd-pchm21:30
ghostmansd-pcnot sure of qemu, as supports it, though21:30
ghostmansd-pclet me check it21:30
ghostmansd-pcperhaps it can be compiled but not run21:31
ghostmansd-pcgdb knows it...21:31
ghostmansd-pcstepi does something strange, though; after entering `stepi` at address 0x20000000 I somehow end up at 0xe4021:33
programmerjakesounds like an illegal instruction trap21:35
ghostmansd-pchttps://lists.gnu.org/archive/html/qemu-devel/2016-11/msg01003.html21:36
ghostmansd-pcyes, looks like an illegal instruction, `add` instruction works OK with `stepi`21:40
ghostmansd-pchttps://gitlab.com/search?search=addg6s&group_id=3038080&project_id=11167699&scope=&search_code=true&snippets=false&repository_ref=master&nav_source=navbar21:42
ghostmansd-pcno luck21:42
ghostmansd-pclooks like we'll be the pioneers21:43
richardwilbur[m]Interesting how the qemu-devel message from Nov 2016 mentions addg6s and a whole slew of BCD instructions.  Did they just not implement them in qemu?21:48
ghostmansd-pcif I got correctly, they're talking of risu21:49
ghostmansd-pcI've checked https://git.linaro.org/people/pmaydell/risu.git21:51
ghostmansd-pcthat's all they have...21:51
ghostmansd-pc$ grep -nri addg6s .21:51
ghostmansd-pc./ppc64.risu:56:# format:XO book:I page:110 v:2.06 addg6s Add & Generate Sixes21:51
ghostmansd-pc./ppc64.risu:57:ADDG6S PPC64LE 111111 rt:5 ra:5 rb:5 00010010100 \21:51
ghostmansd-pcnot much, it looks like they simply generate the instruction, which kinda expected, since risu means "random instruction sequence generator"21:52
ghostmansd-pcOK, I think I'll take another look at addg6s tomorrow21:53

Generated by irclog2html.py 2.17.1 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!