Thursday, 2023-03-30

*** josuah <josuah!~josuah@> has quit IRC00:52
*** josuah <josuah!~josuah@> has joined #libre-soc00:53
programmerjakelkcl: note that you broke src/openpower/decoder/isa/
programmerjakeso the test needs to be updated02:41
programmerjakefound a weird bug, for some reason when decoding nego. the decoder says CR0 isn't written afaict (cr_out == NONE/0), get_cr_out returns None, False04:00
programmerjakedetails in
programmerjakeah, found it after 1h debugging the decoder...I missed that the cr out column in the csv should have been CR0, but it's set to NONE, fixing that.04:59
*** ghostmansd[m] <ghostmansd[m]!> has quit IRC06:18
*** ghostmansd[m] <ghostmansd[m]!~ghostmans@> has joined #libre-soc06:19
*** ghostmansd[m] <ghostmansd[m]!~ghostmans@> has quit IRC09:52
*** ghostmansd[m] <ghostmansd[m]!> has joined #libre-soc09:53
*** josuah <josuah!~josuah@> has quit IRC10:35
*** josuah <josuah!~josuah@> has joined #libre-soc10:35
*** ghostmansd[m] <ghostmansd[m]!> has quit IRC11:22
*** ghostmansd[m] <ghostmansd[m]!~ghostmans@> has joined #libre-soc11:24
*** guest_ <guest_!> has joined #libre-soc11:24
*** guest_ <guest_!> has quit IRC11:37
*** ghostmansd[m] <ghostmansd[m]!~ghostmans@> has quit IRC11:40
*** ghostmansd[m] <ghostmansd[m]!> has joined #libre-soc11:40
lkclahh good catch - but wait did you remember to re-run "pywriter noall simplev"?11:41
* lkcl checking myself11:41
lkcli modified the behaviour of setvl in simplev.mdwn, and although i did actually run "pytest-3 -v -n 8" in the openpower/decoder/isa directory i can't remember if i properly checked the results, whoops11:43
lkcli only have a vague recollection of doing so11:43
lkcl  File "", line 775, in test_svstep_inner_loop_8_jl11:44
lkcl    self.assertEqual(sim.svstate.vl, 12)11:44
lkclah there we go11:44
lkclok i'll sort that.11:44
lkclit's down to changes in behaviour of setvl11:44
lkclah wait... no...11:45
lkclit's down to changes in *DCT*.11:45
lkclmode 2 is now deprecated11:45
lkcli'll have to pick a different function11:45
lkclwasn't expecting that11:45
* lkcl wheeee :)11:48
*** guest_ <guest_!> has joined #libre-soc13:31
sadoon[m]markos: I can't find any documentation on how to tell mini-buildd to build certain packages, the only format it seems to accept is *.changes, any idea how to deal with this?14:41
sadoon[m]The manual shows an example of sending a *.dsc file which would be what I need but it's not accepting that..14:42
markosmini-buildd is just like the normal buildd in that aspect, you have to upload a source package (with it's *.changes and *.dsc files) to a particular upload directory14:47
markosit will constantly scan this directory, when it finds any new package it will send it to the first available builder in the farm14:47
markosofc first it will check gpg signatures, if dependencies are satisfied etc14:48
sadoon[m]See the thing is, there's no .changes because I have no changes over the original sources14:51
markoschanges is produced when you build the source package14:51
markoseg. with dpkg-buildpackage -S iirc14:52
sadoon[m]Ah so it doesn't do something like "apt source" by itself?14:52
markosbut I think you can tell it to just pick the .dsc and rebuild a package14:52
sadoon[m]It keeps giving me errors when I do14:53
markosit's been a few years since I last worked with mini-buildd14:53
markoswhat kind of errors?14:53
sadoon[m]Well to start14:53
sadoon[m]"Not in *.changes format!"14:53
sadoon[m]When using dput and dput-ng14:53
markosyes, dput is the tool that is doing the upload, like dupload, etc14:54
sadoon[m]So it doesn't accept .dsc at all even though there's an example of that in the manual14:54
sadoon[m]Similar error when using mini-buildd-dput14:54
markosit's 2 different things14:55
markosthe dsc is part of the source package itself14:55
sadoon[m]Yeah I got that part, my problem I guess is with the expectation14:56
markosand it basically says "the source package consists of the following (2 or 3) files, here are the checksums/sizes"14:56
markosthe changes file though is the descriptor of the actual upload, and it can consist of a source and/or multiple binaries for one or more architectures14:56
markosso with a changes file you can upload in a single move all binaries as well -if you can14:57
markoshowever that's discouraged14:57
markosand now only source uploads are accepted14:57
markoslet me check a bit14:58
sadoon[m]Unironically my janky script that rebuilds a list of packages (by default fetches the full repo) is more helpful in this case which is.. Funny.14:59
markoscheck the advanced topics "Porting packages (“automatic no-changes ports”)"14:59
markosbasically you don't want to compile a new package, but you want to "port" existing packages, already in the archive to your local buildd15:00
sadoon[m]Alright, building the hello package to test15:07
sadoon[m]Not a terrible option but still needs to be scripted from my end :(15:07
sadoon[m]Unless.. I use the mirror I have and just apply the API call to literally every .dsc :D15:07
markoswell, this is really not to trigger mass-rebuilds15:09
sadoon[m]It's the only option afaict though?15:11
markosI think what we ended up doing is indeed uploading each of the core packages in sequence and triggering a rebuild15:11
markosso yes15:11
markoswe scripted that15:12
sadoon[m]Not a terrible idea and they should be parallelized15:12
markosand we only did it for "only" about 3k packages15:12
sadoon[m]And not even very difficult to script15:12
markospackages would not build until all the dependencies were satisfied anyway15:12
markosbut in order for this work there is a tricky part15:13
markosin the beginning you will have the "target" repo, which is the sffs one15:13
markosinitially this will be empty15:13
markosin order for any package to build, you will need to have a source repo with the original ppc binaries15:13
markosotherwise no dependencies will be satisfied15:14
sadoon[m]Of course15:14
markossorry, source *and* binary repo15:14
markosso for each package built you will need to have 2 binary repos available15:14
markosthe sffs one with high priority15:14
markosand the original ppc one to satisfy the dependencies15:14
markosas packages get build the sffs one will start filling15:15
sadoon[m]So the one you're building will be used as the high priority one, makes sense15:15
sadoon[m]So it pulls from there when it can15:15
markoshowever after a point- when vital packages are built- sffs repo will be sufficient15:15
markosat which point you will have to remove the extra ppc one15:16
sadoon[m]Yup yup15:16
markosand when all is done, just to make sure, trigger a full rebuild of all packages with only sffs repo as source15:16
markosapt source that is15:16
markosat that point you will have a completely sufficient and self-contained sffs repo :)15:16
markosit's a lot of manual work in the beginning but it should get easier as you go15:17
markosnp, glad to help15:17
sadoon[m]So now15:17
sadoon[m]1- script the rebuild with API calls15:17
sadoon[m]2- find out how to apply CFLAGS on all builds15:17
sadoon[m]3- profit15:17
markosI've lost many hours in building debian packages in the past :)\15:18
markosah for 2, you need dpkg-buildflags :)15:18
markosfor most packages that should work15:18
markossome will need manual overrides though15:18
markosthe most tricky ones will be -unsurprisingly- the compilers, gcc/llvm and glibc15:19
sadoon[m]Unfortunately we'll hardly be able to tell which ones don't work until we encounter a "illegal instruction" after the fact15:19
sadoon[m]markos: On gentoo those were fine15:19
markoson gentoo :)15:19
markosgood luck with gcc packages on Debian :)15:19
markosif there is an award for most complicated package in Debian, that should definitely go to gcc, imho15:20
sadoon[m]Ahh :')15:20
markosbut maybe it will be simpler now, who knows15:21
markosif you need help with anything, just let me know15:21
sadoon[m]Thanks again :)15:31
markosyes, portext is the one!15:48
*** ghostmansd[m] <ghostmansd[m]!> has quit IRC15:51
*** ghostmansd[m] <ghostmansd[m]!~ghostmans@> has joined #libre-soc15:51
sadoon[m]But this is super portext!16:29
sadoon[m]There's a warning that it's experimental so maybe not the best option but it's there16:30
*** ghostmansd[m] <ghostmansd[m]!~ghostmans@> has quit IRC16:48
*** ghostmansd[m] <ghostmansd[m]!> has joined #libre-soc16:50
lkcllikely because gentoo is a pure cross-compiler... and consequently gcc itself *disables* most self-testing because it goes "oh you're cross-compiling, let me not run certain tests"18:23
lkclwhereas this is "oh you're running native testing, you asked me to create a gcc native IBM POWER9 compiler, let me just compile up using native IBM POWER9 VSX instructions and then try running that at you"18:24
lkclwhich is whyyyyy the defaults18:24
lkclare always changed18:24
lkclon any native /usr/bin/gcc18:24
lkclto be18:24
lkclof the host18:24
lkclthat you want to run on18:24
lkclyou can't make a cross-compiler the native /usr/bin/gcc on a debian native compiled system18:25
markoslkcl, remember the 20 line function for convolutions? wanna take a guess at how long the NEON implementation is? not finished yet, some tests fail, but almost there18:26
markosactually let me just give the number, 135018:27
markosI used to be a fan of SIMD18:27
*** gnucode <gnucode!~gnucode@user/jab> has joined #libre-soc19:03
lkclmarkos, oh god.19:31
lkcl20 lines of c19:31
lkclcan i possibly lessen the screaming going on inside your brain by first inviting you to write it in RISC-V RVV assembler?19:31
lkclit will help as a transition-point to make the shock of SVP64 much less by comparison19:32
lkcli'm actually serious!19:32
lkclthe good news being it's only a 20 line function19:32
markosI would! if I could actually get a f*cking RVV capable board!19:33
lkclwell, there's always spike19:33
programmerjakewell, if you only need lengths <= minimum guaranteed by V extension, it won't need loops due to unknown VL19:33
markosI'm not using yahe (yet another hardware emulator)!19:33
lkclwhich is properly feature-complete for running user-space binaries19:33
lkclit works - fully. 100%.19:34
markosno, waiting for pypowersim is enough, previously I was doing SVE2 code on qemu19:34
markosnot fun at all19:34
lkclno waiting needed19:34
markosalso no time19:34
lkclit *already has* RVV support - has for years19:34
lkclah that'll do it19:34
markosRVV is on the list, but not right now19:34
lkcli was going to say i'm happy to put a budget towards it but if it's not... yeah the learning-curve *alone* would be too great for the time you've got available19:35
markosI am juggling way too many projects at the moment, am close to a burn out19:35
lkcli forget i know RVV already (to a large extent)19:35
lkcland the spec's *massively* long19:35
lkcl192 instructions or something19:35
lkclso yeah, sadly, scratch that :)19:35
markosyes, I'm having trouble learning SVE/SVE2 as it is, don't need yet another platform right this instant19:36
markoshaving said that, I do intend to learn it, when it's stable19:36
lkclwell the *assembler* is stable19:36
lkclit's the *intrinsics* that aren't19:36
markos... and there is hardware available :D19:36
programmerjakeimho RVV is too much of a distraction to justify our putting a budget to writing a complex algorithm in it19:36
lkclthe assembler's been stable for... err... 2+ years?19:36
lkclprogrammerjake, indeed - that's why i suggested convolution because at its heart it will be literally one instruction in a loop19:37
lkcl(20 lines)19:37
markosalso, the truth is that no matter how I dislike huge implementations, that's what pays the bills right now (NEON/SVE* optimizations)19:37
lkclbut the learning-curve of RVV is itself too steep for markos right now19:37
programmerjakesince we are making PowerISA hw, not RVV19:37
lkclyep, don't knock it19:38
markosfrom that aspect, I'm glad it's complicated as heck because I can charge more :)19:38
markosbut the romantic programmer in me wishes things would go back to simpler architectures, where you didn't need to learn 3k+ page ISA manuals to know which instruction to use19:39
markosthat's why I *loved* SVP6419:39
markosI don't need to remember every f*cking instruction in the ISA to implement an algorithm19:39
markosI am pretty sure I could optimize *all* of libvpx in a 3-4 months compared to more than a year that it took by me *and* a few dedicated Arm engineers/specialists to finish the NEON port19:41
markospossibly less than that once I get the hang of it and have actual hardware or FPGA19:41
markosso imagine the time/cost saving when porting a project to SVP64, that's huge money saving factor for a company19:42
programmerjakewell, at least you don't have to write it in INTERCAL --
markosyeah, I did some COBOL many many years ago, not much fun at all :D19:43
markosthough I hear they are getting paid insane amounts of money19:43
markosmakes sense, they sold their soul!19:44
programmerjakeCOBOL was actually designed to be used, INTERCAL is designed to annoy programmers to no end19:46
markosI remember there was a contest once to create the most annoying/obfuscated programming language19:48
markosI forget which was the winner19:48
markosI remember there was one that used only whitespace characters19:50
programmerjakewell, currently reading how one dialect of javascript is a strong contender:
markosand unsurprisingly it's called 'whitespace' :)19:51
programmerjakeoh, yeah, the programming language named "whitespace" iirc19:51
markosand ofc brainfuck :)19:51
markosbecause some people obviously have too much time in their hands19:52
programmerjakewell, there are implementations of a minsky register machine in game of life...a programming language soo simple it only has increment and decrement19:54
programmerjakethough it's turing complete19:55
gnucodeso you are are working on more graphics related things to power9 right now?  I've jumped in the middle of the convo and have no idea what is being discussed.20:02
gnucodealso the power9 laptop when that ships should be cool.20:02
programmerjakewe were discussing esoteric programming languages, since they'd be more terrible than what markos has to do for his job, create a simd convolution for SVE, which is taking 100x as many insns as svp6420:05
gnucodesimd convolution for SVE  ->  what does that mean?20:06
markosimagine a for loop in C20:06
markosactually 3 nested for loops20:06
markoswith a bunch of math expressions20:06
markosabout 20 lines20:06
markosnot very complicated either20:07
markosnow imagine writing specialized versions of it for a SIMD engine20:07
markosin particular for Arm NEON20:07
markosin C the loop is almost exactly the same, whether one of the loops has 6 or 8 or 12 iterations20:08
markosthe inner loop in particular20:08
programmerjake -- the math that markos is implementing20:08
markosyeah, that's the generic math theory behind it20:09
markosin C the number of filters coefficients does not change the code at all20:09
markosusing NEON, I have to provide 3 different implementations depending if the number of filter coefficients is 6 or 8 or 1220:09
markoseach implementation has to cater for the small widths (<=4) and larger20:10
markosand 20 line C function becomes about 1350 lines with a ton of helper functions20:10
gnucodemarkos: may I ask who is paying you for work on ARM  ?20:10
gnucodesounds like a great time20:11
markosour company is contracted by Arm to do NEON optimizations in a couple of projects, right this time it's libaom, previously it was libvpx20:11
markoswell it works, the resulting code can be many times faster than the original C20:12
markosbut that doesn't mean it's easy to write20:12
markosand definitely not short20:12
gnucodeyou are writing in assembly then?20:12
markosgnucode, basically that's what we do, SIMD optimizations20:13
markosnot even, C intrinsics20:13
markosthough I have done some asm20:13
markosI prefer not to20:13
markosunless I have no alternative20:13
markosin the past it was because the compiler could not use the right instructions20:13
markosie missing intrinsic, buggy code generation, etc20:14
markosbut right now things are much better20:14
markosand that's a fundamental difference with SVP6420:14
markosI've already written a few algorithms in SVP64, mostly small functions but about the same size, 10-20 lines of C20:15
markosand I can translate that to SVP64 in *assembly* at a fraction of the time I have to spend to convert the same algorithm to NEON or SSE/AVX in *C*20:16
markoswhich for me is a game changer20:16
markosnot to mention the difference in debugging, it's much easier20:17
markosand we don't even have hardware yet :D20:17
gnucodeok. spv64 is the extention for openpower that luke and his team created. ok20:18
markosyes, exactly20:18
markosthere is many great features, but in summary as a developer a) I don't have to learn thousands of new instructions, it's essentially a handful of instructions to convert Power ISA to a vector architecture20:19
markosb) you don't need the SIMD implementation to convert to SVP64, it's actually easier -in the cases I tested at least- to start from the C implementation and convert that to SVP6420:20
markosc) you can do clever tricks with the register topology that are just impossible on SIMD, eg. remap, vertical-first, etc20:21
markoson SIMD you have to load and arrange your data in the form you want20:21
programmerjakethat's not always the case though, for utf-8 validation, i did use a simd implementation translated to svp6420:21
markosyes, it's not in every case, but it does apply to many cases20:22
markoswith SVP64 you can load the data and just remap the indices to the structure you want, almost without any data rearrangment, in register or in memory20:23
markosthis is a tricky feature to understand at first, but once you figure out how it works, it's pretty powerful20:23
markosit took me a while at least20:24
markosfinally the fact that it offers 128 registers is just beyond cool!20:24
markosI know lkcl and programmerjake could go on with cool features in SVP64, but for me that's a good start :)20:24
gnucodemarkos: will we see SVP64 in non-power hardware  ?20:27
markosI doubt it20:28
markosbut then again you never know what the future holds, but at least I don't expect it to happen in the next 5 years let's say :)20:28
gnucodemarkos: you expect to see it in power inside 5 years?  That's awesome!20:32
markosI hope, but what I said is I don't expect it to see it in non-power in the next 5 years :)20:34

Generated by 2.17.1 by Marius Gedminas - find it at!