Monday, 2021-12-13

sadoon_albader[mHey guys07:09
sadoon_albader[mlife has been crazy07:09
sadoon_albader[mI'm trying to get back to this and starting to learn things07:09
sadoon_albader[mSo after installing using the devscripts I couldn't find the decoder in the soc repo07:11
sadoon_albader[mOn the main page of the project I found this "python3 src/soc/decoder/power_decoder.py" and so I'm confused, shouldn't the decoder be there after the install process or have I messed up somewhere?07:11
sadoon_albader[mhuh strange, I found it in src/soc/src/soc/decoder07:13
sadoon_albader[mOk I was being stupid nvm07:13
sadoon_albader[mIt's clearly supposed to be there lol07:13
programmerjakemost code on the wiki expect you to be in the root directory of whatever git repo the code is for, so in this case, they expect you to be in soc.git's root directory07:23
sadoon_albader[mnow I know :D07:24
programmerjake:)07:25
tplatenI had a look at https://bugs.libre-soc.org/show_bug.cgi?id=756, this should be easy to fix.09:56
tplatenI guess that the same flags will be used for instruction fetch, so I will have a look at microwatt source code.09:57
lkcli tend to cd soc/src then run editors etc from there10:52
lkclotherwise half the time is spent typing "cd"10:53
lkcltplaten, the idea occurred to me to replace the msr_pr argument with a NamedTuple containing msr.pr, msr.dr, and anything else needed10:54
lkclotherwise we keep jamming in more and more arguments10:54
lkcland you missed a couple on one of the PortInterface unit tests10:54
tplatenI'll fix that10:54
tplatenToo may, I did a grep10:57
tplatenfixed experiment/compldst_multi.py, but not commited yet11:04
tplatenI agree with the NamedTuple thing11:05
tplatenInstead passing msr_pr=0, I propose something like msr = {'pr': 0, 'dr': 0, 'sf': 1}11:32
tplatenI got it11:48
tplatenfrom collections import namedtuple11:48
tplatenMSRSpec = namedtuple("MSRSpec", ["dr", "pr", "sf"])11:48
tplatentest_spec = MSRSpec(dr=1,pr=0,sf=1)11:48
tplatenprint(test_spec)11:48
lkcltplaten: that's the idea12:03
lkclor in the HDL it would be12:03
lkcltest_spec = MSRSpec(dr=msr_i[MSR.DR], pr=msr_i[MSR.PR], ....)12:04
lkcl./experiment/compldst_multi.py:        comb += pi.msr_pr.eq(oper_r.msr[MSR.PR])12:05
lkclmsr_spec = MSRSpec(dr=oper_r.msr[MSR.DR], pr=oper_r.msr[MSR.PR], ....)12:06
lkclMSRSpec namedtuple should probably go iiin.... mmmm..... power_enums.py12:06
lkclor in consts.py12:07
lkcljust after class MSR12:07
tplatenI'll correct that, since it landed in src/soc/experiment/pimem.py12:08
lkclack12:09
lkclit needs to actually be used in the HDL, right down in compldst_multi.py12:09
lkclnot just in the unit tests12:10
lkcland ISACaller could potentially use it although it tends to just pass around all SPRs and regs and gets on with it12:10
tplatenpower_enums.py already imports namedtuple, so I'll add it there, this will be one line change12:11
tplatendone: https://git.libre-soc.org/?p=openpower-isa.git;a=commitdiff;h=218bb6e7b25bbfaaa86727a620d48b3d0ee94bba12:18
lkclfantastic12:19
lkcli'll look at compldst_multi.py12:20
lkclunless you're already doing it?12:20
tplatenNo, I will be working on the other things12:20
lkclok12:20
lkcltplaten, i changed the name in pimem.py of the msr_pr variable to just msr12:27
lkcl-    def set_wr_addr(self, m, addr, mask, misalign, msr_pr, is_dcbz):12:27
lkcl+    def set_wr_addr(self, m, addr, mask, misalign, msr, is_dcbz):12:27
lkcleverything should still work12:27
tplatenI agree, had the same change in mind12:27
lkclah.  compldst_multi.py is now broken because portinterface has msr_pr removed, whoops.12:29
lkclthat could be sorted with a temporary variable-name, which, hmmm, RecordObject won't like, hmmm...12:30
lkclok going into overdrive to sort that out12:30
lkcli think one simple rename has the unit tests sorted (whew)12:33
tplatenhaving a look at the changes from the recent 10 minutes12:38
lkclerm, ermerm... we miiight have had priv_mode inverted in LDSTCompUnit, whoops12:38
tplatenI just saw, PR=problem state not to be confused with priv12:40
lkclyes, they're inverted, as best i can tell from the microwatt source12:40
lkclok just setting pi.virt_mode etc. in LDSTCompUnit.12:43
lkcldo another git pull12:43
lkcli need to re-add MSR to the MMU FSM12:44
lkclcan you do the conversion of msr_pr everywhere to use msr.pr (etc.) instead?12:44
lkclpimem.py12:44
lkcl                self.set_rd_addr(m, pi.addr.data, lenexp.lexp_o, misalign, pr)12:44
lkcl-->12:45
lkcl                self.set_rd_addr(m, pi.addr.data, lenexp.lexp_o, misalign, msr)12:45
lkcli removed MSR from the MMU FSM because it was LoadStore1 that was handling it12:45
lkclbut, ironically, OP_FETCH_FAILED needs to set mmu.priv_mode/virt_mode directly itself, sigh12:46
lkcltplaten, do do another git pull, i had to import MSRSpec in pimem.py12:48
tplatendone12:50
lkclstar12:54
tplatenI updated test_pi2ls.py12:55
lkclgreat12:55
lkcl  File "experiment/test/test_ldst_pi.py", line 48, in mmu_lookup12:56
lkcl    data, _, _ = yield from pi_ld(dut.submodules.ldst.pi, addr, 4, msr_pr=1)12:56
lkclTypeError: pi_ld() got an unexpected keyword argument 'msr_pr'12:56
lkclthere will now be quite a few of these12:56
lkclincluding fu/ldst/loadstore.py (i will do that one)12:57
tplatenI did a grep, there are many of those12:58
lkclindeed :)12:58
tplatencurrently I am fixing test_dcbz_pi.py12:58
lkclloadstore1.py done13:01
lkclwill do pimem.py now13:01
lkcldone13:02
lkcltest_loadstore1.py next13:02
lkclah.  needs an MSRSpec13:02
lkcldone but it doesn't terminate.13:09
lkclneed to investigate13:09
tplatenhaving a look at src/soc/experiment/test/test_ldst_pi.py13:29
lkcltplaten,14:15
lkcl  File "experiment/test/test_ldst_pi_misalign.py", line 30, in <module>14:15
lkcl    msr_default = MSRSpec(pr=1, dr=0, sf=1) # 64 bit by default14:15
lkclNameError: name 'MSRSpec' is not defined14:15
tplatenI'll fix that14:16
lkcldone already14:16
lkclbut it doesn't terminate14:16
lkclthere's something broken now, probably in pi_ld14:16
lkclahh pr is now inverted14:18
lkclthat's what it is14:18
lkclmsr_default = MSRSpec(pr=0, dr=0, sf=1) # 64 bit by default14:19
lkclnot pr=114:19
lkclyep that's it14:20
tplatenI did a git pull14:20
lkcli just fixed test_loadstore1.py as well14:22
lkclfor the invalid exception case, pr must be 114:22
lkclbut for the others, pr=0 which sets priv_mode=HI and then the tests work14:22
markoshello14:24
markoshere on invitation by lkcl :)14:25
tplatenhaving a look at test_loadstore1.py14:36
tplatensrc/soc/experiment/test/test_loadstore1.py:    #ld_data, exctype, exc = yield from pi_ld(pi, virt_addr, 8, msr_pr=0)14:55
tplatenill fix that one14:55
lkclmarkos, oh allo :)15:00
lkclwelcome15:00
markosthank you15:02
markosreading documentation now, there is a lot of information there15:02
lkclindeed.  this is not a small project15:02
lkclminerva rv32 is 4,000 lines.15:02
markoswant to get a working installation first, before I actually commit to anything, I wouldn't want to embarass myself in case I'm in over my head :)15:03
lkclthis one is 150,000 with about 25% completion of the target HDL, and increasing15:03
lkcl:)15:03
lkcla brain-dead way to achieve that:15:04
lkclhttps://libre-soc.org/HDL_workflow/devscripts/15:04
lkclrunning the schroot scripts, *be careful*.15:05
markosyou mention debian 10 as dev environment, is that still so or can debian 11 can be used as well?15:05
lkclit has to operate as root, and makes modifications to...15:05
markos(on ppc64le)15:05
lkclif you do you're on your own15:05
lkclone way round that is to install debian 1115:05
lkclthen use the schroot script to set up a debian 10 chroot15:06
markosok, I'll just create a debian 10 vm on my talos then15:06
lkclthen - and this is where you have to be REALLY careful - bind-mount the home directory (or source subdirectory) into the debian 11 version15:06
lkcli have i think around 4 different schroots at the moment because i run with debian/testing15:06
lkclbut kept python 3.7 around15:07
lkclof course, now, nothing will upgrade, hence i now have some programs (klayout) specifically in a chroot15:07
lkcloo you have a TALOS-II?15:08
markosyup, with about 150G of RAM :)15:08
lkclniiice15:08
tplatenI have a Talos II, and recently started porting Intel Embree to it. Plan to run Embree on the Libre-SOC too.15:09
lkclyet it still falls over if you try to build 3 very large packages at the same time, thanks to the new binutils maintainers ripping out Dr Stallman's "in-memory-only" algorithms in the mid 90s15:09
lkclbecause 640k^H^H^H^H 4GB should be enough for everyone15:09
tplatenThe last remaining ones are src/soc/experiment/test/test_compldst_multi.py and src/soc/experiment/pimem.py15:10
lkclnice15:10
lkcltplaten, ack15:10
lkclRan 1 test in 0.058s15:10
lkclOK15:10
lkcllkcl@fizzy:~/src/libresoc/soc/src/soc$ python3 experiment/test/test_compldst_multi.py15:10
lkclworks fine15:10
tplatenThe test still hangs, I tested with different flags, marked with ? in the commtned out thing15:10
tplatenonly if I undo the comment out thing15:11
lkcltplaten, did you git pull? it works perfectly here15:11
lkclcommit c6cfd46e4c19dfa8c7b36ac39f6c7974df69ff2c15:11
lkcl    fix test_loadstore1.py with MSR=PR/DR15:12
lkcl    for invalid test pr=1 but for others pr=015:12
tplatenI did15:12
lkclhmmm15:12
tplatenonly if i uncomment the following line: # ld_data, exctype, exc = yield from pi_ld(pi, virt_addr, 8, msr=msr)15:12
lkclwhere?15:12
tplatenelse it runs15:12
tplatenline 34115:13
lkclin..15:13
lkclin which file?15:13
lkcltplaten, which source code file?15:15
tplatensoc/src/soc/experiment/test/test_loadstore1.py15:15
lkclok great15:16
lkclbizarre, i don't understand what's going on, it works fine here (no need to set ldst.instr_fault.eq(0)15:17
lkclno need to coment that out15:17
lkclit does make sense though15:17
lkclno it doesn't, because i capture it.  it's only supposed to be set once15:19
lkcl(blipped)15:19
lkclif ldst.instr_fault is left HI it will request multiple MMU_LOOKUPs.15:20
tplatenNow I have a look at src/soc/experiment/icache.py, I began writing a unit test for it this morning15:43
markosin a debian10 lxc right now, I'm getting "Failed sending HTTP request" while cloning the dev-env-setup.git repo, any ideas?17:15
octaviusWhich git clone url are you using?17:16
octaviusThis is the one in my one: gitolite3@git.libre-soc.org:dev-env-setup.git17:16
markoshttps://git.libre-soc.org/git/dev-env-setup.git/17:19
octaviusChange it to the gitolite3 one17:19
octaviusThe one you used is a read-only repo, however the HTTP issue might be due to https (I seem to remember having this issue when I started 3 months ago, on debian 10/buster)17:21
markoshttp works, the git needs access to the repo which I do not have yet17:22
lkcloctavius, marko, yes, gitolite3 (ssh keys) needs agreeing to the charter, first.17:23
octaviusThen stick with the public one until lkcl gets you access ;)17:23
lkclif you're using https you miiight need apt-transport-https17:23
lkclapt-get install apt-transport-https17:24
lkclbut all the URLs in all the documentation should be http://17:24
lkclNOT17:24
lkclnttps://17:24
lkclfor exactly this reason17:24
lkclwhoever thought that encrypting publicly-accessible ("known plaintext") repositories with HTTPS clearly has f***-all idea about cryptography.17:25
lkclsigh17:25
lkcl"eww because it's more sek-ew-aah"17:25
lkcli had a *three* week argument with half a dozen people about that one, once. trying to educate people that putting repos onto HTTPS-only simply makes the client *and server* a high-value hacking target17:26
octaviusI think a lot of people mis-understand the point of encryption and privacy. Ah so what you're saying is if a hacker sees a plain ol' HTTP server, he wouldn't even bother looking at it?17:28
lkclno, they'll go "yippee" with a MITM attack :)17:29
lkclexcept if the packages being downloaded have been GPG-signed with offline-inaccessible keys17:29
lkclwhereupon they'll go "oh poop, forget it"17:29
octaviusDo you mean that if HTTPS is downgraded to HTTP, the hacker can fake the connection?17:30
lkclyes.17:30
octaviusAh right, because it's all plaintext17:30
lkclbut simply having HTTPS does *not* make the data secure... because a hacker can target the server17:30
lkclhack in to it17:30
markosit doesn't have to be a hacker, it's called transparent http proxy  :)17:30
lkcland *still replace the data being downloaded*17:31
lkclyehyeh :)17:31
octaviushehehe17:31
lkclfunniest situation i ever saw of a transparent proxy gone wrong was a china ISP17:31
lkcli visited angband.pl and was getting debian packages being downloaded from http://172.16.21.1/angband.pl/debian/....17:32
lkcli went "wtf" until i realised that they must have been using a home-made transparent HTTP proxy...17:32
lkclas a *perl* script :)17:32
lkcl .pl17:32
lkclwhich their IT staff must have allowed special rules for in their config.17:33
octaviusNiiice17:38
octaviusalso, thanks for another quaint website, I've collected quite a few now :)17:39
lkclthat one sadly the developer kilobyte stopped doing recompiled versions of debian packages without systemd17:45
lkclbecause of devuan17:45
lkclsigh17:45
octaviusAh, he thought it wasn't worth the effort if devuan already has them17:46
lkclcorrect17:46
lkclnot realising that devuan only works on stable-to-stable17:46
lkcland that *his* work worked perfectly well with *standard debian* - even debian/testing systems17:47
octaviusSo does this mean you now have to compile yourself? Or have you found another packager?17:47
lkclno, very annoyingly i was forced into a situation of installing libsystemd017:48
lkclthen the debian sysvinit services make installing systemd redundant...17:48
lkclbut what's particularly irkesome is that systemd continues to "infect" more and more packages17:49
lkclangband.pl was designed to test exactly that situation17:49
lkclto stop people from infecting debian with the assumption that "anything other than systemd init can go f*** itself"17:49
octaviusAh, that library provides (essentially) hard dependencies, without the rest of systemd17:49
lkclcorrect.17:49
lkcland many package maintainers have terminated provision of /etc/init.d/* scripts17:50
octaviusIt does amaze me that having more choice of init systems is generally frowned upon17:50
lkclthey'll stop frowning when they see the CVE mitre list of vulnerabilities for systemd17:50
octaviusIs systemd *really* that good? It provides convinient service creation (if you don't know how to create a symlink)17:50
lkclis Windows NT's service management system *really* that good?17:51
lkclyou honestly think it's a good idea that PID 1 be allowed to control your firewall, network, and filesystem mounts?17:52
octaviusI did struggle when learning gentoo, cause it's usually expected for the user to know about openrc or runit (or other). But that's before I was reading man pages full-time17:52
octaviusYes, sounds like a wonderful idea :P17:52
octaviusPlease give me more bloat, that's exactly why I left win in the first place17:53
lkcltobias: wb_get only copes with 64-bit address offsets18:17
lkclso be careful with that one: i got caught out putting data into the dict at 4-byte offsets then wondered why it wouldn't show up18:19
lkclmarkos, one thing, lauri did request that we not depend on anything outside of the openpower-isa directory18:41
lkclso you should be able to get up and running without soc, ieee754, etc. but you will still need nmigen because the simulator ISACaller depends on it18:42
lkcldecoding Power ISA instructions is such a large task we're not duplicating the code that does it18:42
ghostmansd[m]lkcl, checked your comments in 550, I hope to continue with it on this week18:43
lkclmikolajw is even doing a nmigen-to-c-converter so we can avoid having to maintain 2 (or more) copies of identical Power ISA decoders18:43
lkclghostmansd[m], star18:43
lkclmarkos is a SIMD expert who is evaluating taking over where Lauri was not able to continue due to health18:44
lkclso what you're doing, he could really use (and help test it)18:44
lkclpysvp64asm is the command-line equivalent of what you're doing18:45
lkclwhich replaces "sv.xxxx" with ".long yyyyyy; v30basmop"18:45
lkcli had to add basic support for gnu-as macros into pysvp64asm18:45
lkclghostmansd[m], what you did's a pretty good start18:46
lkcllikewise mikolajw18:47
ghostmansd[m]Thanks! I hope that it will go smoothly.18:50
lkclthere are some (new, draft) 32-bit ops that need adding as well btw18:52
lkclternlogi is one of the latest ones18:52
lkclthey'll all need to go under a special "--experimental" option18:53
lkcllxo, hello, if you're around, what's the procedure for that?18:53
lxoI'm getting into a call in 3 minutes, and I couldn't figure out what the 'that' refers to.  I'll try to figure it out when the meeting is over, some 90 minutes from now18:57
lxobut I wouldn't mind if you'd expand what it is that you mean to ask, lkcl18:57
lxoif you mean what's the procedure for new opcodes to be regarded as experimental in the GNU assembler, I don't think there is one.  each opcode is marked with processor variant identifiers, and that's where this should go IMHO19:03
markoslkcl, thanks I'll keep that in mind19:04
lxolkcl, I'm back.  was that it?20:12
lkcllxo, nice to hear from you. in essence, yes: there must be a "best practice" for enabling experimental (draft) ISA features in upstream binutils,20:24
lkcli was wondering if you knew what it was?20:25
programmerjakeiirc, for risc-v, last i knew, the best practice was experimental instructions never go in upstream binutils20:27
programmerjakethat may have changed since then (several years ago)20:28
lkclmm fun. if so, we're maintaining a binutils fork for NN+ months. joy20:28
lxothere's no general rule that applies to all binutils targets, and ppc is particularly tricky because of all the existing variants, but that's the way to go.  ELF arch variant bits are in short supply so getting one for experimental stuff may be tricky, but I didn't get the notion that this was what the question was about20:29
lxoI'm glad to be able to help, little as it is20:30
lxoI shall try to dial in for a meeting one of these days and catch up.  I seem to have sorted out my internet service, I might even be able to get video up now :-)20:30
lkclwell, svp64 itself is almost certainly going to need one20:30
lkclbut not the computer overheating due to over-active WebRTC? :)20:31
programmerjakewe'll be glad to see you again!20:31
lxoI've got an external "cooling fan" now ;-)20:32
lxolkcl, yeah, I suppose there's a process within OpenPower to assign bits for these purposes, it's not something GNU binutils could/should decide on its own20:33
lkclyeah me too. one at the side and one underneath. with annoying blue LEDs20:33
lkclinteresting. have to raise that on wednesday20:33
lxobut again I think that's not what the question was about, just about binutils-internal marking of instructions, which requires nothing about buy-in from maintainers20:33
lkclwell i'm glad it brought this up too20:34
lxoI'm using usb 3"-4" fans these days20:35
lxoI like to joke that I buy them at a web shop named onlyfans ;-)20:37
markosI just got an error running hdl-dev-repos near the end (I think):20:38
markosIndexError: list assignment index out of range20:38
markosmake: *** [Makefile:10: mkpinmux] Error 120:38
markosis that expected/harmless?20:38
octaviusThe pinmux repo needs to be updated20:38
octaviuslet's see...20:38
octaviusgit submodule update --init20:39
octaviuscall that inside src/soc20:39
markosrerun the cmds, fails in make mkpinmux20:41
octaviusstill the same error?20:42
markosin commit 20ca612b2600530ce901009b3d1b9ef0e05b743820:42
markosyes20:42
markosfull traceback:20:42
markosTraceback (most recent call last):20:43
markos  File "src/pinmux_generator.py", line 101, in <module>20:43
markos    pm, chip = jsoncreate.pinparse(ps, pinspec)20:43
markos  File "/home/markos/src/soc/pinmux/src/jsoncreate.py", line 217, in pinparse20:43
markos    padbank[banknum] = name20:43
markosIndexError: list assignment index out of range20:43
markosmake: *** [Makefile:10: mkpinmux] Error 120:43
markos(btw, do you use any special pastebin here?)20:43
octaviusYeah, this error was fixed. I think the command I gave you is probably missing something20:43
octaviusAh, try running it inside src/soc/pinmux20:43
octaviusThe devscript is still pulling an older commit of the repo20:44
markosgit submodule inside pinmux?20:44
octaviusThat's right20:45
markosteh script it does run gmake gitupdate which calls git submodule update --init --recursive --remote20:45
markosmake not gmake :)20:45
octaviusInteresting. Did running it manually work though?20:46
markosI've never run it before, first time20:46
markosafter make gitupdate and the command you suggested, I tried to run manually make mkpinmux20:47
markosand again the same error20:47
octaviusFound where we discussed it before: https://libre-soc.org/irclog/%23libre-soc.2021-11-29.log.html20:49
octavius* do "git submodule update --init --recursive"20:49
octaviusIf you do a "git log" inside pinmux, can you see the latest commit?20:50
lxoFYI, hours ago, today, I updated my browser's User-Agent overrider.  now, instead of just changing Linux to GNU, I also change x86_64 to libresoc64 :-)20:50
octaviusBut consider your privacy lxo :P20:50
lxothat's a good point, but naming the operating system correctly is enough of a give-away, and not something I'd willingly change back.  so I'm encouraging others to also change it (hint, hint ;-)20:52
octaviusHehehe20:52
octaviusPerhaps when I do have a system powerful enough to emulate it, or even a libresoc devboard20:53
lxoit's just a user-agent overrider, you don't need actual hardware for that ;-)20:54
octaviusAh, so we create a little army of "libresoc64"'s >:)20:55
markos808847daa1773d7636179ea5d49e99b6c7d3a18c20:55
markos(inside pinmux)20:55
octaviusyep, that's latest20:55
octaviusstill breaking?20:55
markosI didn't fix anything so yes :)20:56
octaviusOk just tried it myself20:58
octaviusin soc dir:20:58
octaviusgit pull20:59
octaviusor git pull origin master20:59
octaviusThen inside pinmux:20:59
octaviusgit pull20:59
octaviusgit pull origin master20:59
octaviusNow I'm on latest20:59
octaviusRepeated git pull's because I wasn't sure which one worked XD21:00
lxolong story short, one of the banks I use demands lusers to install spyware of the browser identifies as Linux (or Windows, MacOS, etc), but I was told this could be bypassed by setting the OS name in user agent to FreeBSD.  I found out GNU also worked ;-)21:00
markosoctavius, sorry, same result21:00
markoslatest git commits21:01
markosit has an untracked directory ngi_router inside pinmux21:01
markosmake it's related?21:01
markoss/make/maybe21:02
octaviusDamn, I'm not sure what it is. I know this error was fixed at some point, but I'm getting it as well when I try to build ngi_router21:04
octaviusthe ngi_router directory is a result of this:21:05
octaviuspython3 src/pinmux_generator.py -v -o ngi_router -s ngi_router21:05
octaviusIt is created when given the peripheral spec (what's connected and where to the chip pad21:06
programmerjakeI'd expect the actual default user agent for firefox on libre-soc's hardware to be similar to "Mozilla/5.0 (X11; Linux ppc64le; rv:84.0) Gecko/20100101 Firefox/84.0"21:06
lxo*nod*21:07
lkclmarkos, comment that out for now (ngi_router)22:25
lkclwe updated pinmux recently as you can probably tell22:26
lkcland/or ignore it22:29
lkclpinmux outputs has nothing to do with the simulator22:30
lkclor22:30
lkclgo back to this commit:22:30
lkcld96f737c0a53dde98306052222:31
lkclin the pinmux subdirectory22:31
lkclor22:32
lkclgit pull in soc22:32
lkclthen cd pinmux22:34
lkclgit checkout d96f737c0a53dde98322:34
markosok, that worked22:54

Generated by irclog2html.py 2.17.1 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!