Sunday, 2023-05-07

lkclsorry programmerjake i actually really like the idea of auto-reading markdown for field-offsets: it's just that it's literally the 4th completely disparate system for field offsets introduced into the codebase01:05
lkclthings like that need to be discussed and planned01:06
*** gnucode <gnucode!~gnucode@user/jab> has quit IRC02:48
programmerjakeso, if/when we need to parse the latex field definitions, we can maybe use the parser i wrote as a base, but not now.05:55
*** ghostmansd[m] <ghostmansd[m]!~ghostmans@> has joined #libre-soc05:56
programmerjakeit's more designed for when you need to parse lots of fields, where error checking is helpful cuz errors are much more why the pseudocode parser has nonzero error checking (though it is too close to zero imho)05:57
programmerjakeas i said, i went overboard...sorry05:58
programmerjakeoh, also:
programmerjakehopefully it is 100% open, and not the 75% that is amdgpu06:00
*** ghostmansd <ghostmansd!~ghostmans@> has joined #libre-soc08:45
ghostmansdHi folks, I'm back from flight and vacation, and managed to check
ghostmansdApparently I was confused, because I assumed that minmax has RA_OR_ZERO, and thought that binutils need to switch to RA0 operand08:49
ghostmansdBut this is not the case: minmax instruction does not have RA_OR_ZERO input selectors08:50
programmerjakehmm, maybe it should? i added (RA|0) to the pseudocode but didn't think about the csv...08:52
programmerjakeadded to todo list in #105708:57
programmerjakeapparently luke thought yes:
ghostmansdI can handle it09:10
ghostmansdJust needed a confirmation :-)09:11
ghostmansdSo, in minor_19.csv...09:12
ghostmansd----000011,ALU,OP_MINMAX,RA,RB,NONE,RT,NONE,CR0,0,0,ZERO,0,NONE,0,0,0,0,0,0,RC,0,0,minmax,MM,,1,unofficial until submitted and approved/renumbered by the opf isa wg09:12
ghostmansdshould be replaced with09:12
ghostmansd----000011,ALU,OP_MINMAX,RA_OR_ZERO,RB,NONE,RT,NONE,CR0,0,0,ZERO,0,NONE,0,0,0,0,0,0,RC,0,0,minmax,MM,,1,unofficial until submitted and approved/renumbered by the opf isa wg09:12
programmerjakeyeah, probably. do run the unit tests, since i only know they work w/o RA_OR_ZERO09:14
ghostmansdif I create a separate branch, CI will pick it, right?09:15
ghostmansdOK, pushed into branch09:15
programmerjakethough it takes 40min to run since my server has a really old cpu09:15
programmerjakeif you run pytest locally it'll likely be much faster09:16
ghostmansdwhich test do we have for it?09:16
ghostmansdI can run it on talos09:16
programmerjakeif you just want to run the minmax test file by itself, it should be <1min, probably 5min if you run all tests in parallel09:18
programmerjakein order to run tests in parallel, you'll need pytest-xdist, icr if it's installed by default by our scripts09:19
ghostmansdanything I should run before this test?09:20
programmerjakemake, probably09:20
ghostmansdpywriter, or something similar?09:20
programmerjakemake covers all of that09:20
ghostmansdaha, I knew there's something I miss!09:20
programmerjakeI optimized it so make should take <30s09:20
programmerjakesee the .gitlab-ci.yml for the commands to run09:22
programmerjakemostly just make and:09:22
programmerjakeSILENCELOG='!*,default' pytest -n auto -v src/openpower --maxfail=509:22
ghostmansdI see that we install binutils-powerpc64-linux-gnu binutils-powerpc64le-linux-gnu09:25
ghostmansdLikely we should use dev-env-setup/binutils-gdb-install script instead09:26
ghostmansdLaunched pytest09:29
ghostmansdHow to install pytest globally in chroot?09:30
programmerjakeor just install it for your user09:31
programmerjakepython3 -m pip install --user pytest-xdist09:31
ghostmansd============================================================================== 5 failed, 187 passed, 4 skipped, 1 xfailed, 245 warnings in 161.46s (0:02:41) ===============================================================================09:32
ghostmansdWill check master now09:32
programmerjakeyeah, something broke, master only should have 2 errors09:35
programmerjakesomething broke on your branch or in your chroot09:35
programmerjakei'd guess you're missing something in your chroot, you may want to just wait for ci to finish09:39
ghostmansdOK :-)09:40
ghostmansd1 question: I see that openpower/isatables/RM-1P-2S1D.csv is generated09:40
ghostmansdWhy do we commit it, then?09:40
programmerjakeidk, cuz that's what luke decided?09:41
programmerjakeoh, nvm, i know why...because we want to be able to see it on the wiki and the wiki doesn't run svanalysis.py09:42
ghostmansdk then09:42
programmerjakesince it's technically part of the svp64 specification09:42
ghostmansdyeah seems you're right about configuration09:43
ghostmansdmaster also has a lot of failures09:43
ghostmansdI'll wait for CI then09:43
programmerjakedo note that ci only reports the first 5 errors, so if it says 5 errors there may be many more09:45
programmerjakei chose to have it do that because otherwise the whole error report goes past the 4MB log length limit where gitlab just ignores all further output09:46
programmerjakemaking it really hard to see what failed09:47
lkclprogrammerjake, yes. i like the idea, the commit is recorded so can be picked up later09:54
lkcl nohup pytest-3 -v -n 809:55
lkclif you set to -n auto it can overload your machine09:55
lkcl>  use dev-env-setup/binutils-gdb-install script instead09:56
lkclyes... and update all documentation to match09:56
programmerjakeoh, my desktop picks n=32, the build server picks 809:56
lkcldon't recommend to people that they should run "-n auto" on personal machines without warning them of the consequences09:57
lkclon my laptop it puts such a high loadavg that it becomes unresponsive09:57
programmerjakewell...generally builds are expected to use all cpu cores, right?...09:57
lkcli'm forced to limit those as well09:58
lkclwhich is why i remove all occurrences of "make -j $nproc" from all build scripts whenever it is added09:58
lkclwhat is reset_xflags()?09:58
lkcland why is it not present?09:59
programmerjakemaybe you should just run your whole chroot under cpuset (or whatever the name is), that way all builds that reasonably expect to use all cores can, just they only see however many cores you selected09:59
lkcltoo much effort both for me and for new developers10:00
programmerjakereset_xflags is a function from powerisa v3.1 that i haven't yet added10:00
lkcldo add functions as stubs if they are used, or comment out the unit tests10:01
lkcli don't mind code being added as long as it isn't actually in use10:01
programmerjakei left it uncommented so you can see if fpscr is recognized, i'll fix it soon10:01
ghostmansdFAILED src/openpower/decoder/isa/ - AttributeError: reset_xflags10:02
ghostmansdFAILED src/openpower/decoder/isa/ - AssertionError: 2 != 110:02
ghostmansd= 2 failed, 349 passed, 75 skipped, 19 xfailed, 748 warnings in 1877.08s (0:31:17) =10:02
ghostmansdSeems like the same as for master10:02
lkclgo for it10:03
programmerjakehopefully much faster than how long you've taken to fix test_sv_load_dd_ffirst_excl10:03
programmerjakethx ghostmansd!10:03
ghostmansdthanks to you, actually :-)10:04
ghostmansdmy job was quite a simple one10:04
lkclyes i know. that's a significant intrusive change in ISACaller which i'm... how-to-say-delicately... "working up the energy to tackle"10:04
lkclVLi=0 the load or store must *not* actually happen10:05
ghostmansdbringing the CI is a real job and time-saver10:05
ghostmansdpushed to master10:05
ghostmansdnow I can update binutils10:05
lkclbut in the case of a load it does have to happen but just not actually go into the register10:05
lkcli.e. you read the data from memory, then test it10:05
ghostmansdfor the record, we also have setvl affected by the same issue10:05
ghostmansdso I'll update it too10:06
lkclif the test fails you must not actually store in RT.10:06
lkclghostmansd, yes please, but watch out for unit tests on that one... oh wait... hang on...10:06
lkclin setvl it does things slightly differently10:06
ghostmansdlkcl, I mean updating setvl in binutils10:06
lkclthe pseudocode actually looks at the contents of RA (the 5-bit) by itself10:07
ghostmansdnothing to be changed for openpower-isa10:07
ghostmansdit's just binutils having RA instead of RA010:07
ghostmansdfor minmax, both code bases were incorrect10:07
ghostmansdfor setvl, only binutils's the sucker10:07
programmerjakewell, as i recommended before: all element ops that succeed should always write all their outputs, all that VLi=0 should do is adjust VL *after* that element writes to RT10:07
lkclthat's VLi=1 mode10:08
lkclwhat you've described as "recommended" is exactly what VLi=1 is for10:08
lkclVLi=0 is, by definition, "all elements that succeed on a test"10:09
lkclVLi=1 is, by definition, "all elements tested"10:09
programmerjakewell, icr the polarity of VLi but i mean the one that ends up with VL=<index-of-last-successful-op> rather than VL=1+<index-of-last-successful-op>10:09
lkclyou're asking for VLi=0 to be defined as being "identical to VLi=1"10:10
lkcli.e. make it pointless10:10
lkcland that in turn destroys key functionality10:10
programmerjakeno, i'm asking for VLi=0 to be defined as identical to VLi=1 except afterwards it subtracts one from VL10:11
lkclno. not happening10:11
lkclLD/ST is a special case that needs a little more thought to implement10:11
programmerjake*not*: runs last op then tests results and proceeds to not write them10:11
lkclArithmetic/Logical/CR-ops VLi=0/1 is perfectly fine and logical10:12
lkclthe answer is no and that was final the last time you raised this.10:12
lkclghostmansd, am re-running the unit tests locally to double-check10:13
ghostmansdFor the record, I've been looking at this job on CI10:13
programmerjakeif a load page-faults, then it *failed* hence shouldn't write RT. if a load reads a zero, it *succeeded* and should write RT=0, but changing VL -= 1 is still useful10:13
lkclah you are confusing page-fault with data-dependent-fail-first10:14
lkclplease do read the specification.10:14
lkclit is slightly alarming the number of times i have had to ask you to do that (read the specification)10:14
lkclsections 1.6 and 1.710:15
programmerjakewell, you said reads 0 then proceeds to not write it, which sounds like data-dependent-fail-first to me10:15
lkclthat's *not* the same as a page-fault.10:15
lkclDD-ffirst is the *actual data returned by the load*10:16
lkclwhereas fault-first is whether the underlying hardware caused a LD/ST-related excception that *prevented* the actual data from *being* returned10:16
lkclit's explained very clearly in the specification10:17
programmerjakewell, imho, for ddff, that's wrong...the load succeeded, so it should succeed 100% including writing just returned 0 so future element ops aren't run10:17
lkclno it should not.10:17
programmerjakefor avoid-page-fault fail-first, not writing is 100% logical10:17
lkclthe load succeeded *and if VLi=1* the write to the registers should be performed10:17
programmerjakewell, i disagree10:18
lkclbut if the load succeeded *and if VLI=0* then if the test **OF THE DATA LOADED** fails the test this should **NOT** proceed10:18
lkclthat's because you don't understand it.10:18
programmerjakeit unnecessarily makes element ops a bit more complex10:18
lkclplease take the time to ask the questions "why is this necessary, what have i missed?"10:19
lkclDD-FF gates whether the *write to the regfile* takes place10:19
lkclnot whether the entire operation should, as an atomic entity, take place in its entirety yes or no10:20
programmerjakei understand how you designed it, i just think you designed it in a way that makes it unnecessarily more complex10:20
lkclthat's VLi=110:20
lkclit's very simple to implement.10:20
lkclan OoO Micro-Architecture simply inserts the DD-FF testing in as a Shadow Hold on the write operation10:20
lkcljust like on predication10:20
lkcland branches10:20
lkcland exceptions10:20
lkcland interrupts10:21
lkclso in hardware terms it is very simply exploiting pre-existing hardware that you absolutely have to have anyway10:21
lkclbut that also tells you why it exists: it stops the *write to regfile* from happening10:22
lkclstops the entire operation from happening10:22
lkclyou're thinking as a software developer in terms of "this instruction is atomic", not in terms of a hardware architect10:22
lkclthe difference is huge. in terms of say FP exceptions, anything that overflows can be prohibited from damaging the regfile10:23
programmerjakewell, imho stopping the write isn't super useful in sw, what is the useful part is adjusting VL down by one10:23
lkcli just literally described a use-case where that opinion is invalid. it was crossover though10:24
lkclto continue10:24
lkclif you are forced to detect the overflow within a hot-loop it reduces the performance of the hot-loop10:24
ghostmansdlkcl, is the test OK?10:25
lkclwhereas if instead VLi=0 stops the damage to the regfile from occurring, but provides a "notice" at least (if Rc=1), then a special branch to overflow-handling-scenario code can be performed10:25
lkclghostmansd, 76% completed10:25
ghostmansdcan I remove remote minmax_ra0 branch?10:25
ghostmansdAha, staying tuned, please ping me when it's completed10:26
lkclghostmansd, might as well, i'm just being cautious10:26
lkclthe reduction in instruction count is quite significant10:26
ghostmansdthat's perfectly reasonable10:26
programmerjakewell, for strcat-like things, we specifically want the write to occur to write the nul, but we don't want VL to include the nil...10:27
lkcloh - *and* there are scenarios where VLI=1 are also necessary, such as linked-list-pointer-chasing as well as strncpy10:27
lkclthat's taken care of by following up with an sv.cmpi. please read the strncpy example and take the time to understand it10:28
programmerjakewhere you have strcat followed by something that needs to operate on the non-zero bytes10:28
lkclprogrammerjake, taken care of already10:28
lkcl  50                 "sv.cmpi/ff=eq/vli *0,1,*16,0",  # compare against zero, truncate VL10:29
lkclbut yes, making absolutely sure that all use-cases are covered is part of the NGI Search grant responsibility10:30
programmerjakeactually, the fail-first needs to be on the load, since the code you have rn will read out of bounds10:30
lkclhope like hell it's all good10:30
programmerjakecmpi after reading out of bounds is still reading out of bounds10:31
lkclit can't read out of bounds10:31
lkclit can only test from 0..VL-110:31
programmerjakebut VL-1 may be beyond the bounds of the input str, reading out of bounds with the sv.lbzu/pi10:32
lkclhow? the length (VL) is restricted to the bounds (n)10:33
programmerjakeso imho that example is broken when you can have page faults and sigsegv for reading beyond the terminating nul byte10:33
lkcl(using CTR)10:33
lkclyou're again confusing page-faults (LD/ST-**FAULT**-First)10:33
lkclwith **DATA**-dependent Fail-First10:34
programmerjakestrncpy just guarantees the output is valid for n, the input may be shorter10:34
lkclplease READ the specification.10:34
lkcland come back to me with a summary of what you have read10:34
lkclsection 1.6 and 1.710:34
programmerjakewhen the input is shorter, that sv.lbzu/pi *will* read beyond the input, because it isn't a page-failt-avoiding fail-first10:34
lkclno it will not, because VL will have by that point been reduced down10:35
lkclghostmansd, all good10:35
lkcl  41                 "mtspr 9, 3",                   # move r3 to CTR10:35
lkcl  61                 "sv.bc 16, *0, -0xc",           # decrement CTR by VL, stop at zero10:35
programmerjakewhere, the ddff is *after* the load10:35
lkclyes it is.10:36
ghostmansdI was afraid I'll have to update this comment:
lkcland VL will be within bounds on the LD10:36
lkclas defined by CTR10:36
lkclit's good that you're checking because this does need checking10:37
programmerjakeno it won't, because VL/CTR is set by the output length, strncpy doesn't know the input length until it reads the terminating nul10:37
programmerjakeand since that load isn't ffirst it will read past the nul10:38
lkclahh ok so a difference between input-length (implicit) and output-length (n)10:38
programmerjakethis is before it goes around the loop even once10:38
lkclyes it will read past the nul, and that then allows the sv.cmpi to perform the test10:38
lkclbut it will *not* overrun the *output* length, which is critically important10:39
lkclso the stbu does not result in data-corruption10:39
programmerjakeno, it'll read past the nul and segfault, not even getting to the cmpi, when the nul is right before an unmapped page10:39
lkclno, it will *not* read past the nul and segfault.10:40
lkclthat is prevented by *LD-Fault-First*.10:40
lkclplease for goodness sake for the third time *read the spec*10:40
programmerjakewhy? it is *not* ld-fault-first10:40
programmerjakeit's just plain ld10:40
lkcl(i realise i haven't set it to /ff because there's no actual way to test LD/ST-faulting in the simulator)10:41
lkcl(ISACaller doesn't actually have /lf - sorry not /ff)10:41
programmerjakeah, so that's why10:41
programmerjakeunimplemented :)10:42
lkclindeed. so much to do, sigh :)10:42
programmerjakeshould probably add a comment that it should be ld-fault-first10:42
lkclgood idea10:42
programmerjakeonce that's done, we both agree it should work :)10:43
lkclthe problem with implementing Load-Fault is that it requires completely different behaviour from ISACaller in order to accurately test it10:45
programmerjakewell, we could fake faults for unittests and implement page-table-walking later...10:45
lkcloh page-table-walking is already implemented10:46
lkcli had to add it for testing against microwatt-linux-5.710:46
programmerjakee.g. add: if addr not in defined_addresses: return page_fault()10:46
lkcland the mmu.bin microwatt unit test10:46
*** ghostmansd <ghostmansd!~ghostmans@> has quit IRC10:46
lkclwhich is behaviour that needs to be added as options to ISACaller at startup10:47
lkcl"force-an-exception-if-a-load-or-store" occurs is a good way to do it, tbh10:48
lkclas a new option to ISACaller constructor10:48
lkcli need breakfast.10:48
*** ghostmansd[m] <ghostmansd[m]!~ghostmans@> has quit IRC11:04
*** ghostmansd[m] <ghostmansd[m]!~ghostmans@> has joined #libre-soc11:05
programmerjakeoh, btw, "add stub reset_xflags function" -- no point, the pseudocode uses like a dozen new functions, just wait till i implement them11:07
*** ghostmansd[m] <ghostmansd[m]!~ghostmans@> has quit IRC12:41
*** ghostmansd[m] <ghostmansd[m]!~ghostmans@> has joined #libre-soc12:44
*** choozy <choozy!> has joined #libre-soc13:06
*** octavius <octavius!> has joined #libre-soc14:35
octaviuslkcl, sorry for the lack of communication recently, been busy with other things. Looking at the microwatt sim today, and I forgot to mention the trace waveform file.14:42
octaviusIn the makefile, --trace flag passed in to generate the VCD trace file.14:42
octaviusIn the microwatt-verilator.cpp, the if VM_TRACE variable is defined, the trace file *should be* updated twice per clock cycle, and the setup code resembles the example in the Verilator manual (section "How do I generate waveforms (traces) in C++?").14:42
octaviusThe issue I'm seeing is that the 'microwatt-verilator.vcd' file is created when the microwatt-verilator is first run, but afterwards it is not updated. Converting this empty vcd file to fst and viewing in gtkwave shows only the state of signals at time 0.14:42
octaviusWhat I'm expecting instead, is for it to update as the simulation runs.14:42
octaviusHave I missed something? I haven't passed any flags to the microwatt-verilator binary (other than the hello_world.bin to load).14:42
*** ghostmansd[m] <ghostmansd[m]!~ghostmans@> has quit IRC15:13
*** ghostmansd[m] <ghostmansd[m]!~ghostmans@> has joined #libre-soc15:14
lkcloctavius, you have to wait for a file-flush16:20
lkcland/or pass in options to have verilator write out with fsync.16:20
lkclyou'll need to look up the manual.16:20
lkcland/or as usual ask around on the interwebs16:21
octaviusOk, then I'll leave it out of the tutorial for now. I was also able to run micropython.16:23
lkclawesome! it's a little slow isn't it but functional16:26
lkclhmmm... creating vcd traces is kinda important16:27
lkcli have vague recollections of leaving things running for several minutes16:27
lkclthen getting bored and looking up the verilator compile-time options to write out trace files more frequently16:27
*** tplaten <tplaten!~tplaten@> has joined #libre-soc16:38
octavius"creating vcd traces is kinda important", yeah, you're right. I'm looking through some tutorials16:44
octaviusThe wiki page is starting to look more comprehensive:16:45
octaviusIf you want, I can join some of this back to the original page. The reason I made a new one is just to focus on microwatt only (so the user sanitises the flow first)16:45
*** octavius <octavius!> has quit IRC17:15
tplatenstatus update: currently reading
*** choozy_ <choozy_!> has joined #libre-soc18:08
*** choozy <choozy!> has quit IRC18:14
lkcltplaten, the PDF version may be easier but ultimately it's the same thing18:55
lkclalthough a couple (new) pages need adding.18:55
lkcloctavius: merging it back is better, as it's effectively a duplicate (and therefore a maintenance headache)18:57
lkclas a general rule if there's an existing page please don't duplicate it18:57
tplaten433 pages19:07
tplatengoing to have first look tomorrow19:56
*** tplaten <tplaten!~tplaten@> has quit IRC20:14
ghostmansd[m]tplaten, there's an unreplied email about binutils. Looking forward to get an answer.20:51
markos_ghostmansd[m], he did send to the list that there was a bounced mail to you21:15
markos_also got an error sending a mail to21:15
markos_<>: host[]21:15
markos_550-5.7.26 This mail is unauthenticated, ..........21:15
markos_So I'll use the mailing list or the chat instead21:15
ghostmansd[m]Does it happen on the direct mail?21:16
markos_no idea, I can send a test mail to you if you like21:16
ghostmansd[m]I've never met this thing before. What does it even mean, bounced email?21:17
markos_well, error, 550 means the user was unreachable/wrong mailbox21:17
markos_bounced mail is something else21:17
markos_means that the server is unreachable so the mail could not be delivered to the server -because of server being temporary offline, but that does not mean the mail is wrong/non-existent21:18
markos_this definitely reached the server, but was rejected21:18
ghostmansd[m]Hm. Perhaps this is caused by the fact that my email is in fact, not ghostmansd? But these are the same in Gmail.21:18
ghostmansd[m]And I always use undotted form21:18
markos_just sent a test mail to this address21:19
ghostmansd[m]markos, I got it :-)21:19
markos_not from a gmail account21:19
markos_ok so the mail works21:19
ghostmansd[m]subj is test mail21:19
ghostmansd[m]Hm. Pretty damn strange.21:19
ghostmansd[m]Ok so Tobias replied somewhere?21:20
markos_so something on tplaten's message21:20
markos_tell you what, I'm going to forward his mail to both of you21:20
markos_btw, are you subscribed to the list?21:21
ghostmansd[m]Ok, ping me after you send it :-)21:21
ghostmansd[m]Not sure which one you mean21:21
markos_you should be, fwiw, it's not that heavy traffic21:21
markos_actually libre-soc-dev21:22
ghostmansd[m]Yes, I think so21:22
ghostmansd[m]At least I sometimes get updates on the project21:22
ghostmansd[m]For today I have a message from Tobias about daily kan-ban for 07 May21:23
markos_that's the one :)21:23
markos_that's where I copied the text from :)21:24
ghostmansd[m]Ah OK21:24
ghostmansd[m]That's the one I missed21:24
ghostmansd[m]I assumed I'd get some reply in the chain where I raised the question :-)21:24
markos_well, now that I've restablished comms, I can quietly disappear in the background :)21:26
ghostmansd[m]Thank you :-D21:30
ghostmansd[m]The reason I'm somewhat curious is that I've been developing this, I guess, for a year or so already, and I need to at least be aware of what Tobias is going to show or present21:31
*** choozy_ <choozy_!> has quit IRC23:57

Generated by 2.17.1 by Marius Gedminas - find it at!