Wednesday, 2021-09-22

lkclLas[m], the heuristic for running appropriate unit tests i went through it with ghostmansd and kylel quite recently05:23
lkclit needs becoming a bit familiar with the dependency hierarchy of the code, but basically the further up towards "unused" or "little-used" leaf node code, the less unit tests you need to run [with semi-confidence]05:25
lkclthere's a few "big" tests (25 mins) and some REALLY big tests (several days)05:26
lkclthe heuristics allow you to avoid running those on every single commit05:26
lkclone trick:05:27
lkcl* write completely independent code that is totally isolated from all other code.05:27
lkcl* commit it freely even without unit tests (not really encouraged, that) as long as it has no syntax errors05:28
lkcl* write completely independent unit tests, again committing those freely without reservation05:28
lkcl(i mean, *not* even using branches, because there's absolutely zero possibility of "damage")05:29
lkcl* once and ONLY once you are happy that the (independent) code actually works, work out how to "hook" it into "live" code05:30
lkclthis last part may need some preliminary / preparatory code-morphs before the "live" code is ready to "accept" the (up until then independent) code05:30
lkclit's an incremental process and it's one we're literally going through right now with the Test API05:31
lkclso if you track the commits made across the soc and openpower-isa repos here
lkclyou can get a pretty clear idea of how *not* to need to use branches05:32
lkclwhich i detest05:32
lkclhaving had quite significant work *completely* disregarded, multiple times, because i was told "f*** off and use branches, moron"05:33
lkcli'm not about to inflict that onto anyone else.05:33
lkclthat said, the XLEN work which required very careful review, used an xlen branch, and it worked extremely well.05:34
lkclthat involved cherry-picking of something mad like *100* small commits, but it set up a good review process. similar to how gerrit works, but without needing a horror-show-of-web-service05:36
programmerjakelkcl, if your still up, do the budget-sync changes sound good? (feel free to review/test tomorrow)05:42
lkclprogrammerjake, i'm a schrodinger cat at the moment :)05:43
programmerjakehmm, sleep and look in the morning?05:44
cesarlkcl: Could you please check that you have the latest pia? Maybe reinstall it? The error that I'm getting originates from pia.11:53
cesarprogrammerjake: Commit 720517cf0ff588aebcf97a5eadc566ab680b0791 causes a regression in src/soc/fu/mul/test/test_pipe_caller.py12:04
cesar^ pia commit (in case it wasn't clear)12:08
lkclcesar, yes i have that commit. i'm not going to update pia in case it breaks12:15
lkclcesar, i have an idea, if you can submit a set of test values which fail we can check it manually and add a regression test which doesn't use pia.15:22
lkclkylel, neat trick with the sets15:26
kylelthanks.  you saved me some time leading to sets.  quite fortunate how sets work for that application.  I wanted to avoid a bunch of "if not in" type logic15:58
lkclshockingly, guido van rossum advocated for their removal about 8-10 years ago16:32
lkclLas[m], i meant to ask: what's the kinds of things you can do / have done? and of course, what would you *like* to do?16:57
lkclthe idea being, there, to find you something useful you'd enjoy16:57
Las[m]lkcl: It's wrt. Nix packaging16:58
lkclkylel, am just endeavouring to slice HDL away from Sim, first step, decouple Sim in setup_tst_memory by passing it the test.mem16:58
Las[m]Preferably I'd upstream the code so more people can make use of it16:58
Las[m]Related: I got my FPGA yesterday. It would be great if I could test the Libre-SOC on it. Is there a wiki page for this?16:58
lkclnice! which one?16:59
Las[m]Some ECP516:59
Las[m]No, unfortunately not16:59
Las[m]Summer of Nix is a timed event, and that would ship in November16:59
lkclan orangecrab or something like that?16:59
Las[m]Which is way too late16:59
Las[m]let me check16:59
lkclit needs to be one that's supported by litex16:59
Las[m] IIRC17:00
lkclahhh exccellent17:00
lkclhey those were in stock again. great17:00
lkclok then it's brain-dead simple17:00
lkclrun the commands at lines 14 and 1517:00
Las[m]How do I know it works though?17:01
lkclconnect to the serial console with minicom17:01
lkcl1 sec17:01
lkclif you see the litex bios on the serial console, you know it's good.17:02
lkclif you want to get a little bit more sophisticated, you can grab N.E.Other FT2322 and wire it up to the pins we're using for JTAG TAP17:03
lkclwhich... you can see all the Stuff On How To Do That at...17:03
lkclthat's a *completely separate* FT232 from the one used *by* the actual VERSA_ECP517:05
lkclso you *will* end up with the two getting horribly confused when using openocd :)17:06
lkclbecause it goes, "oh, errr, you have *two* FT232s plugged in, i have no idea how to discern between them, which one's the programmer for uploading FPGA bitstreams and which one's for bit-banging JTAG to the Libre-SOC core? pffh, i have no idea, barf"17:07
lkcli solved that by, um, disconnecting one of them at a time :)17:07
lkcli now have a USB hub with individual power buttons17:07
lkclbut, if you don't plan to run the JTAG TAP testing (which i have to say is kinda cool), then you don't need to be concerned about it17:08
lkclthe FT232 on the VERSA_ECP5, you just run minicom -D /dev/ttyUSB0 and you're good.17:08
Las[m]I'll do that17:13
* lkcl thumbs-up17:15
lkclghostmansd, your daughter (and family) all ok?17:15
ghostmansdYes. My wife has returned and was able to pick up things as they were for two days more, but, frankly speaking, by the time she returned the daughter was already healthy. :-) They only had to wait for two days more until kindergarten allows them to return there.17:18
ghostmansdThat said, I'm somewhat in a hurry at work after these two weeks.17:19
lkclyehyeh understandable17:19
ghostmansdThings that deadline has been approaching to... Well, they're almost deadlined. :-)17:19
lkclwell what are you talking to *me* for then?? :)17:19
ghostmansdThat's since a man for sure needs some other joys besides deadlines! :-D17:22
programmerjakecesar: idk how that breaks it since all that commit does is add an implementation for mulli17:59
cesarIt is returning a ValueException, before it was returning an AttributeException.18:03
programmerjakeah, the issue is that mulli needs immediate set in InstructionInput's constructor, the other mul ops have that input in rb instead18:04
programmerjakebefore that commit was trying to call pia.mulli but it didn't exist hence the attributeerror18:06
programmerjakei'll try to fix test_pipe_caller18:13
programmerjakecesar lkcl: I fixed it!19:00
lkclwha-hey :)19:00
programmerjaketook longer than expected since I had to re-figure out how to extract the immediate from the decoder19:01
lkclPowerDecoder2 is a biiig piece of work19:02
lkclwhich is why i don't want to be duplicating it in c19:03
lkclhence the nmigen-to-c compiler19:03
programmerjakefor #699 i don't know that giving all the money to luke and I is any better...anyone else have any opinions? cesar?19:06
kylellkcl, unit_test failure works. woot19:07
programmerjakei asked cole about #699's budget and he said he doesn't care about getting money for it19:12
lkclok that's a relief because i scaled everything and realised it'd be like, below EUR 30.  so took everyone but you and i off19:20
lkclkylel, w00t :)19:20
cesarI don't mind, I can't really approach the amount of work both of you put into this. Also, there is plenty of other funded work still.19:21
programmerjakek, thx!19:22
* lkcl just realised i have an error in adde, no idea why, it's off by one, consistently19:22
lkclthx cesar19:22
cesar"There are 2 hard problems in computer science: cache invalidation, naming things, and off-by-1 errors."
lkclcesar, lol. someone put that into the bugzilla quips/quotes19:26
programmerjakeidk, it might already be there19:28
programmerjakeyeah, i added that quip when we first setup bugzilla19:29
jncesar: and DNS combines all of them :)19:37
lkclkylel, not-so-big split, kinda obvious now the other pieces are in place19:42
lkclsplit out the functions for HDL and ISACaller(Sim)20:20
* lkcl
* lkcl
lkclon a bit of a youtube music video spree :)20:53
programmerjakeoh, well in that case:
programmerjakeinteresting video by Kurzgesagt21:08
lkclkylel, if you're good with the expectedFailure test (committed?) the next most logical incremental step would be to add an "ExpectedState"21:09
lkclprogrammerjake, informative21:09
lkcllike, "hey uhh you can build all the electric cars you want, but that doesn't take care of the fact that the effing roads are made with bitumen!"21:10
* lkcl face-palm21:10
kylel lkcl, was just about to ask where you wanted test_states dumped into? openpower/test?21:11
lkcltest_states... ehn? :)21:11
lkcloh right21:11
lkclyes, if it's "general purpose", yes absolutely.21:12
lkclyou saw what i did with soc simple/test/ 1 HDL 2 Simulator 3. Compare21:12
lkcl3. Compare needs to be what moves to openpower/test, yes21:13
lkclslowly though we need to morph towards being able to specify a command-line argument to all of *and* to test_issuer.py21:13
lkclmerging them all and allowing "--sim", "--hdl", "--microwatt", "--qemu" etc etc etc etc21:14
programmerjakedoes that work with unittest infrastructure? that's what I'm depending on to parallelize testing...21:18
lkclif you try to run qemu parallelised i'd consider that to be quite quite insane21:19
programmerjakeif the tests can't just be run by `python3 -m unittest`, that is problamatic21:19
lkclit would fire up multiple qemu instances which would quickly overwhelm system resources21:19
programmerjakewell...only if you have a small amount of ram...21:20
kylelspeaking of qemu, that will be interesting given how dang slow the controller is.21:20
programmerjakeif you have 32GB like I do you should easily be able to run several qemu at once21:20
lkcli suspect we need to do a close examination of pygdbmi21:21
lkcland a (temporary, hopefully) fork of it21:21
programmerjakeI'd expect each qemu to only need <1GB for just unit tests...if not we need to adjust qemu's ram allocation to more sane levels21:21
lkclit relies on a timeout for basic communication21:21
kylelyeah, there is another library out there that does async gdb which I assume would be faster but haven't looked much into it.  I was amazed I got it to work.21:21
programmerjakewell...timeouts work just fine with 24 parallel invocations...21:22
lkclrather than looking for an expected prompt and using proper buffering21:22
kylelI think that one simple test I did took maybe around...wanna say a minute for one instruction.21:23
programmerjakeyou may want to also look into lldb's api...iirc it can be directly used from python21:23
programmerjakeno need to parse stdio for that21:24
programmerjakelldb should also be able to connect to qemu's built-in gdbserver21:26
lkclurrrr swig auto-generated. looovely :)21:27
lkclargh i'm breaking s***.21:32
lkclwow awesome!21:46
lkclan actual (severe) python 3.7 segfault!21:47
richardwilbur[m]<lkcl> "split out the functions for..." <- youtube:  “This video contains content from SME, who has blocked it in your country on copyright grounds.”21:47
lkclrichardwilbur[m]: doh21:47
lkclgreeeeat :)21:47
lkcli'm deeply impressed to have gotten an actual segfault out of python for something that should be "working"21:48
lkcli.e. not a 3rd party module21:48
programmerjakewell...getting a segfault is pretty easy...just mess with ctypes21:50
lkclyeah that's expected.21:50
programmerjakeor maybe with python's ir21:50
lkcllike calling assembler and going "oh i fink i will just obliterate the stack and expect everyfink to be hunky dory"21:50
lkclthis is an extremely rare refcounting error21:51
lkclthanks to crossing over from yield21:51
lkclmeeting 8mins21:52
lkclcesar, lxo, programmerjake &21:53
programmerjakeyeah, just a min21:55
lkclok, we're waiting for ya :)22:02

Generated by 2.17.1 by Marius Gedminas - find it at!