| ghostmansd-pc | Holly Molly, it seems I can finally run tests on Talos | 05:15 |
|---|---|---|
| ghostmansd-pc | and, actually, unlike on my VM, these tests don't take forever to run | 05:15 |
| ghostmansd-pc | I was able to run test_pipe_caller_long.py | 05:17 |
| programmerjake | :) | 05:25 |
| ghostmansd-pc | hm, on test_issuer.py there are still issues... | 05:28 |
| ghostmansd-pc | File "/home/ghostmansd/.env/lib/python3.9/site-packages/libresoc_nmutil-0.0.1-py3.9.egg/nmutil/gtkw.py", line 8, in <module> | 05:28 |
| ghostmansd-pc | from vcd.gtkw import GTKWSave, GTKWColor | 05:28 |
| ghostmansd-pc | ImportError: cannot import name 'GTKWColor' from 'vcd.gtkw' (/home/ghostmansd/.env/lib/python3.9/site-packages/pyvcd-0.1.7-py3.9.egg/vcd/gtkw.py) | 05:28 |
| ghostmansd-pc | that's from venv; if I run vanilla python3, I have the same output, with python residing in /usr/local | 05:28 |
| programmerjake | hmm, you have a version of pyvcd that's too old, GTKWColor first shows up in v0.2 in this commit: | 05:53 |
| programmerjake | https://github.com/westerndigitalcorporation/pyvcd/commit/d142a67ba6e582457013c1a38c408fcf338a7f5b | 05:54 |
| programmerjake | oh, you were gone when I posted a reply... | 05:56 |
| programmerjake | reposting | 05:56 |
| programmerjake | hmm, you have a version of pyvcd that's too old, GTKWColor first shows up in v0.2 in this commit: | 05:56 |
| programmerjake | https://github.com/westerndigitalcorporation/pyvcd/commit/d142a67ba6e582457013c1a38c408fcf338a7f5b | 05:56 |
| programmerjake | hope that helps | 06:00 |
| ghostmansd | programmerjake: thank you! Does it need Luke's intervention, or I can install it locally? | 07:12 |
| ghostmansd | With the former, perhaps both venv and system Python will work, but you never know for sure... | 07:13 |
| programmerjake | with a venv (assuming it's in your home directory), your free to install whatever python stuff you please .. it won't break anyone else since they use their own venvs or the system python | 07:16 |
| programmerjake | you should be able to install it with (in the venv) `pip install pyvcd==<version>` where <version> is whatever version you want... | 07:19 |
| programmerjake | you probably want 0.2.4 which is the latest version | 07:19 |
| programmerjake | https://pypi.org/project/pyvcd/ | 07:20 |
| programmerjake | so, you shouldn't need to ask luke to upgrade pyvcd in your own venv | 07:21 |
| programmerjake | hopefully that helps! | 07:21 |
| lkcl | ghostmansd, use python3 setup.py install --user | 07:27 |
| lkcl | venv will stop you from accessing the already-installed packages in ~/.local | 07:27 |
| lkcl | and require you to re-run all of the commands that i already ran in order to get the tests to run, for you | 07:28 |
| lkcl | programmerjake: venv is like chroot, it cuts you off from everything that you've done outside of the "chroot" | 07:29 |
| programmerjake | yup, i know... | 07:29 |
| lkcl | consequently, recommending to ghostmansd to use venv forces him to go through the already-actioned install process yet again | 07:29 |
| programmerjake | hmm, i thought he had done that in a venv already... | 07:30 |
| lkcl | no, it didn't work | 07:30 |
| programmerjake | oh, :( | 07:31 |
| lkcl | so i used python3 setup.py develop --user | 07:31 |
| programmerjake | venv works just fine for me... | 07:31 |
| programmerjake | it makes it so i can install whatever i like without having to worry about conflicting versions messing up system programs | 07:32 |
| programmerjake | install --user still has conflicts | 07:33 |
| programmerjake | but at least it's better than sudo pip install... | 07:33 |
| lkcl | wtf nmigen 0.2 is installed?? | 07:37 |
| lkcl | f****g f*****g pip, pissing me off | 07:37 |
| lkcl | a whole stack of things had been installed by someone running as root - by pip - which included pulling in nmigen 0.2 | 07:41 |
| lkcl | i reaaaally intensely dislike pip and how it carries out unauthorised downloads without your consent | 07:43 |
| programmerjake | oops...more reasons to use a venv... | 07:45 |
| lkcl | ghostmansd: ok, i've removed the f****d-up pip-installed version 0.2 of nmigen, and the libresoc-* packages that pip3 brought in without informing anybody | 07:45 |
| lkcl | i'm not sure it would actually help, because pip3 would *still* - under the venv - pull in unauthorised downloads | 07:45 |
| lkcl | like, exactly as has happened in /home/ghostmansd/.env/lib/python3.9/site-packages | 07:47 |
| programmerjake | we need *one* script that does the full setup process, so newbies can just run setup.sh and it creates a venv, clones git repos, does the whole setup process, finally leaving you with a fully working setup. kinda like rustc's x.py | 07:47 |
| lkcl | ghostmansd: you've gone and run things in a random order, which has resulted - a second time - in pip3 going and randomly f*****g about and carrying out unauthorised downloads | 07:48 |
| lkcl | don't use venv | 07:48 |
| lkcl | i repeat again: don't use venv | 07:49 |
| lkcl | the order of the installation of the python dependencies is CRITICAL | 07:49 |
| lkcl | and is listed EXPLICITLY in the scripts. | 07:49 |
| lkcl | if you run them manually - which i can tell from the results that you've been doing that - you MUST run the installs in the EXACT AND PRECISE ORDER listed in the dev-env-setup scripts | 07:50 |
| lkcl | and in NO OTHER ORDER | 07:50 |
| lkcl | if you try to run any of the setup.py scripts from libresoc in the wrong order, pip3 will go, | 07:50 |
| lkcl | "oh, you don't have one of the dependencies, let me go off randomly to get that for you, from some random version you have no idea about, and i won't ask your permission before doing so" | 07:51 |
| programmerjake | sorry for contributing to the confusion | 07:51 |
| lkcl | consequently, by running "python3 setup.py develop" in, for example, the "soc" directory, it went: | 07:52 |
| lkcl | * oh you don't have nmigen, let me get that for you. hmm, what version shall i install? oh, i'll get nmigen 0.2 for you | 07:52 |
| lkcl | * oh, what does nmigen 0.2 need? oh, that requires pyvcd 0.1.7. let me terminate all and any possibility of upgrading that because i've installed nmigen 0.2 for you. | 07:53 |
| ghostmansd | lkcl: thanks for clarification, now it's clear | 07:53 |
| lkcl | * oh, you don't have nmutil, let me just grab that off of pypi | 07:53 |
| lkcl | then.... | 07:53 |
| lkcl | you go | 07:53 |
| lkcl | cd nmutil | 07:53 |
| lkcl | python3 setup.py install | 07:53 |
| lkcl | and it goes | 07:53 |
| lkcl | "ARGH, i can't do this because it conflicts with the version already installed by pip" | 07:54 |
| lkcl | actually it's much worse than that | 07:54 |
| lkcl | it goes, | 07:54 |
| ghostmansd | However, in my defense, I must say: it's not clear what and how to install on Talos. On system you own, you can simply run whatever script we have in dev-env. | 07:54 |
| lkcl | "ARGH, i can't do this, because there's a version of nmigen already installed by pip, which conflicts with the version needed" | 07:54 |
| lkcl | it gets worse | 07:54 |
| lkcl | you then cd to the nmigen git repo | 07:55 |
| ghostmansd | So yes, I _assumed_ it'd be fine to follow the install from setup.py, because, well, that's what was mentioned in discussion. | 07:55 |
| lkcl | and... | 07:55 |
| lkcl | you get the idea | 07:55 |
| lkcl | indeed | 07:55 |
| ghostmansd | Was there a specific order mentioned? I don't recall. | 07:55 |
| lkcl | welcome to pip3 hell | 07:55 |
| lkcl | i didn't think to mention it because it's in the dev-env-setup script | 07:55 |
| lkcl | so if you stop using the venv and just use the user-local-installed stuff which i did a few hours ago for you, it'll be fine | 07:56 |
| ghostmansd | Ok, I'll use it, thank you! | 07:56 |
| ghostmansd | Again, in case I ever need to re-do the thing, is there a clear sequence for a new developer? | 07:56 |
| lkcl | ls -altr /home/ghostmansd/.env/lib/python3.9/site-packages to see the full extent of the horror-show created by pip3 | 07:57 |
| ghostmansd | Because currently the docs is "call lkcl, he's kind and hopefully will help you" | 07:57 |
| lkcl | includes nmigen-0.2 (unauthorised download) | 07:57 |
| lkcl | normally it would just be hdl-dev-repos-virtualenv | 07:58 |
| lkcl | https://git.libre-soc.org/?p=dev-env-setup.git;a=blob;f=hdl-dev-repos-virtualenv;hb=HEAD | 07:58 |
| lkcl | that's kept in sync with the equivalent (system-wide) hdl-dev-repos script | 07:59 |
| lkcl | you can compare the two and see that they're near-identical | 07:59 |
| programmerjake | well, we should add that to the docs on the wiki...first I heard of that | 07:59 |
| programmerjake | i just did a roll-your-own venv for my computer | 08:00 |
| programmerjake | admittedly, i started before we had dev-env-setup | 08:00 |
| lkcl | it's been there for.... 15 months. tobias created it | 08:01 |
| lkcl | https://git.libre-soc.org/?p=dev-env-setup.git;a=history;f=hdl-dev-repos-virtualenv;hb=HEAD | 08:01 |
| lkcl | i've never been particularly keen on virtualenv, i lean more towards a preference for doing full chroots | 08:03 |
| programmerjake | k | 08:03 |
| lkcl | that's just me | 08:04 |
| programmerjake | i'm kinda partial to how cargo does it: versions (and sources) are declared in each project's toml file, and the code is supplied by cargo, independently of every other project. no system-wide conflict-ridden mess. too bad python went for the interpreter equivalent of dll hell | 08:06 |
| lkcl | it's nothing to do with python | 08:07 |
| lkcl | it's entirely pip3 | 08:07 |
| programmerjake | well, pip is de-facto python | 08:07 |
| lkcl | before pip3 went ballistic, things were explicit. | 08:07 |
| lkcl | no, it's not. | 08:07 |
| lkcl | "the interpreter" != "a package used to install other packages" | 08:08 |
| programmerjake | what I meant was, for the python ecosystem, pip is the de-facto method of obtaining dependencies | 08:09 |
| lkcl | debian got this right. if there's more than one package to be pulled down, it prompts you unless you add the "-y" option | 08:09 |
| lkcl | if there's just one package it just does it. | 08:10 |
| lkcl | yes, that's true | 08:10 |
| programmerjake | and python has to live with the consequences | 08:11 |
| lkcl | fortunately - very fortunately - at least pip3 detects system-wide packages | 08:11 |
| lkcl | yeah. | 08:11 |
| lkcl | i'm not impressed by some of the decisions made by the Python Software Foundation in recent years | 08:11 |
| programmerjake | cargo is a lot more like nix -- dependencies are taken from a key-value store and assembled together exactly how the program your compiling desires, no system-wide conflicts | 08:13 |
| programmerjake | admittedly i haven't looked at nix too closely so i could be incorrect | 08:14 |
| lkcl | i suspect that when packages stabilise and end up being installed by distros that cargo will run into exactly the same problem faced by pip3 right now | 08:14 |
| lkcl | the problems come from interaction between stably-declared releases and developer versions | 08:15 |
| lkcl | that's really the source of the problem for pip-installed packages | 08:15 |
| programmerjake | one other nice part is it will let you use multiple major versions of a dependency in the same program | 08:15 |
| lkcl | a developer declares "my software works solely and exclusively with this version X of package Y and this version X1 of package Y1" | 08:16 |
| lkcl | that's exactly what pip3 allows. | 08:16 |
| lkcl | don't think for one second that what cargo does is "new" or hasn't been done many many times by many package-installers | 08:16 |
| lkcl | the problem comes when a developer wants to use latest-and-greatest-out-of-git | 08:17 |
| programmerjake | nope, not new, just a really good idea | 08:17 |
| lkcl | if the declared versions in the cargo manifest happen not to match those, the developer is screwed | 08:17 |
| lkcl | just like - exactly like - the situation with pip3 manifests | 08:18 |
| programmerjake | then, one just uses a git dependency in your toml file rather than using pip's approach of saying "i want v0.3" and installing it yourself behind pip's back | 08:18 |
| lkcl | i know everyone (including me) keeps blaming pip3 itself, but actually the source of the problem is an interaction between stable "packaged" releases and developer versions | 08:18 |
| lkcl | which *you* as the developer have to override | 08:19 |
| lkcl | *you* as the developer have to be the one to manually and explicitly edit that in | 08:19 |
| lkcl | the exact same thing is perfectly possible with pip3 manifests: it's just that nobody actually does it because it's such a pain in the ass | 08:20 |
| programmerjake | pip's problem is it *doesn't* have a way to say "i want the version from commit xxx of git.example.com/repo.git" | 08:20 |
| lkcl | yes, it does. | 08:21 |
| lkcl | it's just that nobody uses it - or is aware of it | 08:21 |
| lkcl | plus, it's a pain in the ass | 08:21 |
| programmerjake | hmm, never seen that | 08:21 |
| programmerjake | well, we should use it then...or is it hopelessly broken? | 08:21 |
| lkcl | using that, you have a maintenance headache at the developer level | 08:21 |
| lkcl | it's a maintenance nightmare | 08:22 |
| lkcl | every single commit in one repo, we now have to update five other hash tags in pip3 manifests?? | 08:22 |
| lkcl | i don't think so | 08:22 |
| lkcl | i'm doing 30+ commits a day on a good day. | 08:23 |
| programmerjake | well, you can declare it to depend on a branch instead (at least in cargo) | 08:23 |
| lkcl | no way in hell i'm wasting time updating pip3 manifests 30+ times a day | 08:23 |
| programmerjake | well, if the dependency is "master@git.blah/repo.git", and not a commit hash, it is better | 08:24 |
| programmerjake | idk if pip supports that | 08:25 |
| programmerjake | cargo does | 08:25 |
| lkcl | it should do. i've never used it. | 08:25 |
| lkcl | always preferred to work with "setup.py develop" on the most-used (in-development) packages | 08:26 |
| programmerjake | this all is why I was advocating for a monorepo...everything is guaranteed to work together (assuming we test commits). pip dependencies could then just be "../dependency/" (or however pip phrases getting something from the local fs) | 08:27 |
| programmerjake | we won't have to worry about version mismatch | 08:28 |
| lkcl | yyeh, look up freeswitch for how that turns out | 08:29 |
| programmerjake | links? google gave nothing apparently relevant for "freeswitch monorepo" | 08:31 |
| lkcl | 100+ dependencies that they pulled in and "maintained" themselves | 08:31 |
| lkcl | then wondered why people complained when they failed to keep the security patches up-to-date on their "maintained" dependencies | 08:32 |
| programmerjake | pulling in external non-libre-soc dependencies is not what I'm advocating, i'm advocating putting all libre-soc hdl-related code in a single repo | 08:33 |
| lkcl | got themselves into a total nightmare situation where they *couldn't* update the dependencies with security patches, because they hadn't kept the core software up-to-date, and were critically relying on hopelessly out-of-date versions | 08:33 |
| lkcl | yes. no. | 08:33 |
| programmerjake | it worked out just fine for llvm | 08:34 |
| lkcl | a monorepo basically declares to users, "under no circumstances should you even remotely consider using the libraries we create, for your own project" | 08:34 |
| lkcl | yeah - we're not llvm. | 08:34 |
| lkcl | llvm is a monolithic project | 08:35 |
| lkcl | for a monolithic single-purpose project ("a compiler"), a monorepo is a natural fit | 08:35 |
| programmerjake | no, it doesn't...we can create mirror repos that follow subprojects of the monorepo, just like llvm | 08:35 |
| lkcl | nmutil is not in *any way* actually related to soc, at *all* | 08:36 |
| lkcl | it's the direct equivalent of chisel3's io library | 08:36 |
| programmerjake | llvm, isn't very monolithic anymore...they've grown to have a bunch of not-closely related stuff | 08:36 |
| lkcl | chisel3's io library is very deliberately not included in chisel3 for exactly the same reasons that nmutil is not included in either nmigen *or* soc. | 08:37 |
| lkcl | the ieee754fpu library is, likewise, very specifically intended *not* to be something that forces people to think, psychologically, of it being "solely exclusively for use with and by libre-soc" | 08:38 |
| lkcl | they're designed as completely independent autonomous libraries for *general-purpose* usage | 08:38 |
| lkcl | a monorepo psychologically prevents and prohibits people from considering that even as a possibility | 08:39 |
| programmerjake | well, since they're submodules of soc.git, it means they're still kinda a monorepo with extra painful syncing steps | 08:39 |
| lkcl | forcing contributors to have to download massive packages and hunt through tens to hundreds of thousands of lines of code just to get on tiny library that they actually want? | 08:39 |
| lkcl | yep, i know | 08:40 |
| programmerjake | as i said----git mirrors of parts of the monorepo | 08:40 |
| lkcl | except because i do it all the time, it's not at all painful | 08:40 |
| lkcl | i just run the "git update" commands naturally out of habit, without thinking about it | 08:40 |
| lkcl | once we hit a stable release point, everything changes. | 08:41 |
| lkcl | which is why i spent some time a couple months back registering nmutil, ieee754fpu etc. etc. with pypi | 08:41 |
| * lkcl tired. need to investigate a bug for ghostmansd | 08:42 | |
| lkcl | divw fails | 08:42 |
| programmerjake | well, because you have to make sure the submodule is updated in a separate commit, and it's hard to diff across, or have synchronized commits, or have moving code make sense to git, etc, etc, etc | 08:42 |
| lkcl | i know. git submodules are supposed to help with that, but have their own problems | 08:43 |
| programmerjake | or when people forget to push their submodule commits | 08:44 |
| programmerjake | a monorepo solves all of those | 08:44 |
| programmerjake | anyway, good luck with divw, i'm going to sleep | 08:45 |
| lkcl | yeh me too (again) shortly | 08:45 |
| ghostmansd | So for now we have two issues, right? | 08:50 |
| ghostmansd | one in shift and one in arith | 08:51 |
| lkcl | ghostmansd, take a peek at the "depends on" list on 671 https://bugs.libre-soc.org/show_bug.cgi?id=671 | 08:51 |
| lkcl | i just sorted divw | 08:52 |
| lkcl | https://bugs.libre-soc.org/show_bug.cgi?id=683 | 08:52 |
| lkcl | am re-running test_issuer.py | 08:52 |
| lkcl | it'll be another 15 mins and i need to go lie back down | 08:52 |
| lkcl | srad is broken and it's not obvious at aaaalll why | 08:53 |
| * lkcl will be awake again in about 4-5 hours | 08:54 | |
| Las[m] | lkcl: Do you know if there's any specific reason some python modules in Coriolis are in `/etc`? Are you expected to modify them as a user of the software? | 11:28 |
| Las[m] | Or are they supposed to be readonly? | 11:28 |
| lkcl | installed in etc? ??? | 14:01 |
| lkcl | that's definitely a FSH violation | 14:01 |
| lkcl | i'll have to talk to Jean-Paul about that. | 14:02 |
| lkcl | i'd never noticed before. | 14:02 |
| Las[m] | It's the `symbolic.cmos` modules and such | 14:50 |
| Las[m] | e.g. https://gitlab.lip6.fr/vlsi-eda/coriolis/-/tree/devel/crlcore/etc/symbolic | 14:50 |
| lkcl | i will tell Jean-Paul off :) | 17:38 |
| lkcl | it's clear that the "normal" install process is supposed to be in ~/ (homedir) | 17:39 |
| lkcl | ghostmansd, test_issuer.py passed, last night, for the (fixed) divw | 17:39 |
| lkcl | after some missing brackets | 17:39 |
| lkcl | but srad doesn't | 17:40 |
| ghostmansd | So that's was precedence issue? | 17:40 |
| lkcl | yes, in the actual pseudocode | 17:40 |
| lkcl | [0]*XLEN/2 | 17:40 |
| lkcl | was translated to | 17:40 |
| lkcl | concat(0, repeat=XLEN)/2 | 17:41 |
| lkcl | which is of course a list divided by an integer :) | 17:41 |
| lkcl | i fixed it by doing | 17:41 |
| lkcl | [0]*(XLEN/2) | 17:41 |
| lkcl | it's actually valid from a precedence perspective, * has priority over / | 17:41 |
| ghostmansd | Aaah | 17:41 |
| lkcl | :) | 17:42 |
| ghostmansd | Damn :-) | 17:42 |
| lkcl | the srad one is not obvious *at all* what the hell is going on | 17:42 |
| ghostmansd | Do we have a content of registers by the time it failed?... | 17:43 |
| ghostmansd | I think I might reproduce it with test_caller.py | 17:43 |
| lkcl | in the bugreport, yes. | 17:43 |
| lkcl | yes good idea | 17:44 |
| ghostmansd | Does it include initial regs? | 17:44 |
| lkcl | or, make a test_caller_shift.py | 17:44 |
| lkcl | errrr... | 17:44 |
| lkcl | actually i think the bug is "bad" enough that *any* values fail (!) | 17:44 |
| ghostmansd | That's actually better for us :-) | 17:44 |
| lkcl | can i suggest, though, if you want to do "known" values, to simply temporarily patch-out fixedshift.mdwn | 17:45 |
| lkcl | back to its original state | 17:45 |
| lkcl | https://git.libre-soc.org/?p=openpower-isa.git;a=commitdiff;h=1a7e5359c7a4d4bc9ee8e0536f2c01c5f1865cbd | 17:45 |
| lkcl | patch -p1 -r | 17:46 |
| lkcl | errr.... hang on, it just passed | 17:47 |
| lkcl | let me re-run | 17:47 |
| lkcl | ha, passed, irony | 17:48 |
| lkcl | that may just be my fault, not running pywriter noall fixedshift | 17:48 |
| lkcl | doh | 17:48 |
| lkcl | let me re-run test_issuer.py agaaaaaain | 17:48 |
| lkcl | sorry about that :) | 17:49 |
| lkcl | ghostmansd: comparefixed is a next most likely candidate | 17:51 |
| lkcl | https://libre-soc.org/openpower/isa/comparefixed/ | 17:51 |
| lkcl | cmpeqb will need a minor rewrite to use a for-loop | 17:51 |
| lkcl | pretty much exactly the same thing as bpermd which is byte-based, isn't it? | 17:52 |
| lkcl | btw you starting to see why i suggested the XLEN task for you? it's necessary to go over *every* Power ISA instruction | 17:52 |
| lkcl | which gives you a "hands-on" actual practical working knowledge of what's _actually_ in the ISA | 17:53 |
| lkcl | whoops it looks like he got disconnected from the internet, doh | 17:58 |
| lkcl | https://bugs.libre-soc.org/show_bug.cgi?id=671#c58 | 17:58 |
| ghostmansd | Yep, got it :-) | 18:01 |
| ghostmansd | Actually, the more I deal with it, the more I like it | 18:02 |
| ghostmansd | Initially it was quite frustrative, though | 18:02 |
| ghostmansd | Did I get it right that pywriter wasn't run after you fixed off-by-one? | 18:06 |
| ghostmansd | And the error that was in issue is caused by that? | 18:06 |
| lkcl | looks like it | 18:10 |
| lkcl | test_issuer.py completed let me check it | 18:10 |
| lkcl | no errors! all good | 18:10 |
| lkcl | closed it | 18:11 |
| lkcl | suggest https://bugs.libre-soc.org/show_bug.cgi?id=671#c58 as the next batch | 18:11 |
| lkcl | there's only 6 instructions, but they're quite odd | 18:12 |
| lkcl | oh, fixedstore.mdwn as well | 18:12 |
| * lkcl out for a walk | 18:12 | |
| ghostmansd | Ok, in progress now :-) | 18:24 |
| ghostmansd-pc | lkcl: I'm still somewhat confused with fixedstore. What'd be the right change? | 19:05 |
| ghostmansd-pc | let's consider sth, which has `MEM(EA, 2) <- (RS)[48:63]` | 19:05 |
| ghostmansd-pc | 1) MEM(EA, 2) <- (RS)[XLEN-16:XLEN-1] | 19:05 |
| ghostmansd-pc | 2) MEM(EA, MIN(2, XLEN/8)) <- (RS)[48:63] | 19:06 |
| ghostmansd-pc | 3) MEM(EA, MIN(2, XLEN/8)) <- (RS)[XLEN-16:XLEN-1] | 19:06 |
| ghostmansd-pc | https://libre-soc.org/irclog/%23libre-soc.2021-09-01.log.html#t2021-09-01T11:45:44 | 19:07 |
| ghostmansd-pc | https://libre-soc.org/irclog/%23libre-soc.2021-09-01.log.html#t2021-09-01T12:02:20 | 19:08 |
| lkcl | no | 19:18 |
| lkcl | on phone | 19:18 |
| lkcl | just MEM(EA, 2) | 19:18 |
| lkcl | no change to mem width | 19:18 |
| lkcl | part of spec | 19:18 |
| ghostmansd-pc | OK, so only RS[XLEN] change | 19:22 |
| lkcl | yes, correct | 19:28 |
| lkcl | the discussion we had on the mailing list a couple days ago | 19:29 |
| lkcl | basically if XLEN/8 < MEM(EA, {width}) we consider it UNDEFINED behaviour | 19:29 |
| lkcl | Lauri pointed out that it would be an extremely bad idea to "silently" change the behaviour of any of the ld operations | 19:30 |
| lkcl | lw "magically" becomes a different width ==> ExtremelyBad(tm) | 19:30 |
| programmerjake | hmm, i think it might be better to load at full lw width and truncate to elwidth -- just like uint32_t -> uint8_t conversion in C | 19:32 |
| programmerjake | ISA-level UB should be avoided | 19:33 |
| programmerjake | too bad OpenPower is full of UB... | 19:34 |
| programmerjake | they should have picked sign/zero extension for modsw, not UB extension | 19:35 |
| lkcl | yes, that's effectively what is done | 19:44 |
| lkcl | sigh yes | 19:44 |
| lkcl | and then had IBM POWER9 do something completely different | 19:44 |
| lkcl | in between those two (load at full width, truncate to dest elwidth) goes the "saturation" | 19:45 |
| lkcl | in a very-carefully-documented specific order | 19:45 |
| lkcl | in order to cope with possible sign-extension as well | 19:45 |
| lkcl | e.g. sv.lb/sw=16/dw=32/sats | 19:46 |
| lkcl | which err.... means... | 19:46 |
| lkcl | * load a byte | 19:46 |
| lkcl | * extend it to 16-bit | 19:46 |
| lkcl | * then extend it *again* to 32-bit | 19:46 |
| lkcl | something like that | 19:47 |
| * lkcl needs to re-read the spec again https://libre-soc.org/openpower/sv/ldst/ | 19:47 | |
| lkcl | right, ok, some of the LDs are arithmetic, so involve sign-extension. that was it | 19:48 |
| lkcl | so a good example would be... lha | 19:48 |
| lkcl | sv.lha/sw=32/dw=16/sats | 19:48 |
| lkcl | which would be: | 19:49 |
| lkcl | ... wait.... | 19:49 |
| lkcl | sv.lha/dw=16/sats | 19:49 |
| lkcl | which would be.... | 19:49 |
| lkcl | * load at 32-bit (the original 32-bit LD behaviour) | 19:49 |
| lkcl | * perform sign-extension to 64-bit | 19:50 |
| lkcl | * examine whether the 64-bit result as a *signed* integer is greater than 0x7FFF (32767) or less than 0x8000 (-32768) | 19:50 |
| lkcl | * perform signed-saturated truncation if it is | 19:51 |
| lkcl | so the dw=16 tells "sats" what the saturation limits are | 19:51 |
| lkcl | it's quite specific and quite tricky | 19:51 |
| lkcl | has to cover a really wide range of possibilities | 19:51 |
| lkcl | an entire quadrant of which wasn't properly discussed / evaluated until this came up a few days ago | 19:52 |
| lkcl | (whoops) | 19:52 |
| ghostmansd-pc | btw, regarding ldst. there's a couple of either explicitly or implicitly 64-bit instructions, and some are double-word: stq, sthbrx, stwbrx, stdbrx | 19:52 |
| ghostmansd-pc | and also stmw, I guess | 19:53 |
| ghostmansd-pc | I guess we should keep these as is? | 19:53 |
| ghostmansd-pc | their ld* variants we kept as is | 19:53 |
| lkcl | yep skip the double-word ones entirely | 19:54 |
| lkcl | we're not doing them | 19:54 |
| ghostmansd-pc | lmw/stmw are not double words, though, but still are implicitly 64-bit | 19:55 |
| lkcl | they're optional in the "Level 2" Compliance Subset (Embedded Floating-Point) | 19:55 |
| lkcl | yeah we're definitely not doing those. | 19:55 |
| lkcl | if you look closely you'll see those are deprecated | 19:55 |
| lkcl | because they're a massive pain, great idea in 1994 when you're not doing high-performance Multi-Issue Out-of-Order superscalar designs | 19:56 |
| lkcl | but an absolute blasted nuisance if you are | 19:56 |
| ghostmansd-pc | :-D | 19:56 |
| lkcl | even ARM had to learn that one and deprecate their multi-register instructions | 19:56 |
| lkcl | ARM settled for a twin-LD, twin-ST | 19:57 |
| kylel | Well that the install nightmare is over...ready for orders | 19:57 |
| lkcl | kylel: :) | 19:57 |
| ghostmansd-pc | lol, ARM even introduced cmpxchg | 19:57 |
| kylel | Oh, a few things about the venv script... | 19:57 |
| ghostmansd-pc | I might be non-objective, but this ll/sc is a total crap | 19:57 |
| programmerjake | for the double-word atomic load/stores...we still need those since software needs 128-bit atomics | 19:57 |
| ghostmansd-pc | cmpxchg for the win | 19:58 |
| lkcl | kylel, yah go for it | 19:58 |
| ghostmansd-pc | programmerjake: dw-cas? | 19:58 |
| ghostmansd-pc | or generic 128-bit atomics? | 19:58 |
| * lkcl kylel, i just added you to the dev-env-setup gitolite3 write-access list | 19:59 | |
| ghostmansd-pc | iirc most of atomics can be implemented atop of cmpxchg | 19:59 |
| kylel | Just minor differences about checking for sudo, creating src directory, etc. No biggie. | 19:59 |
| programmerjake | in reference to above where lkcl was saying we don't need 128-bit load/stores | 19:59 |
| lkcl | programmerjake: ah. you think we'll need them? | 20:00 |
| programmerjake | not a reply to ll/sc vs cmpxchg | 20:00 |
| * lkcl trying to find them | 20:00 | |
| programmerjake | yes, lots of sw depends on 128-bit atomics for stuff like wait-free linked lists | 20:00 |
| ghostmansd-pc | lkcl: pushed xlen branch with fixedstore; before checking it, I should re-run `hdl-dev-repos`, `hdl-dev-repos-virtualenv`? | 20:00 |
| lkcl | ghostmansd-pc, ah no. | 20:01 |
| lkcl | just re-run pywriter noall fixedstore | 20:01 |
| lkcl | that should do it | 20:01 |
| ghostmansd-pc | > yes, lots of sw depends on 128-bit atomics for stuff like wait-free linked lists | 20:01 |
| ghostmansd-pc | hm, do you mean doubly-linked list? | 20:01 |
| lkcl | by using "python3 setup.py develop" you've requested that python look at the *development* directory | 20:01 |
| lkcl | the actual git clone checked-out source code | 20:01 |
| ghostmansd-pc | singly-linked should be solved (at least in C) as Harris | 20:01 |
| lkcl | you *do not* need to re-run "python3 setup.py develop" multiple times | 20:02 |
| ghostmansd-pc | let's assume I've just ssh-ed to talos | 20:02 |
| lkcl | if however we had used python3 setup.py ***INSTALL*** then that *would* be necessary to re-run | 20:02 |
| ghostmansd-pc | what should be next steps? | 20:02 |
| programmerjake | see isa v3.1 section 3.3.4 for some reasoning | 20:02 |
| ghostmansd-pc | (including git) | 20:02 |
| lkcl | and you've just git committed something from your local machine to the server? | 20:02 |
| ghostmansd-pc | I have it in xlen remote branch | 20:03 |
| ghostmansd-pc | which I can pull | 20:03 |
| lkcl | ok then it's just a matter of "git pull" | 20:03 |
| ghostmansd-pc | and even try rebasing it | 20:03 |
| ghostmansd-pc | so, ssh, then git pull --rebase, then test_issuer.py? | 20:04 |
| lkcl | git pull --rebase | 20:04 |
| ghostmansd-pc | (and also pywriter) | 20:04 |
| ghostmansd-pc | sorry, forgot to mention | 20:04 |
| lkcl | pywriter noall fixedstore (or whatever MDWN file was modified) | 20:04 |
| lkcl | then yes, run test_issuer.py | 20:04 |
| lkcl | or if you want something shorter | 20:04 |
| lkcl | python3 fu/compunit/test/test_ldst_compunit.py | 20:04 |
| lkcl | that should be very quick | 20:05 |
| ghostmansd-pc | `git pull --rebase` -- only openpower-isa, or all I have in src/? | 20:05 |
| lkcl | if there's errors, go back to editing and fix those | 20:05 |
| lkcl | well, given that you don't know whether anybody else has also done some commits, it maaaay be a good idea to git pull on all other repos as well | 20:05 |
| programmerjake | see also the lqarx and stqcx. instructions | 20:05 |
| lkcl | but that's up to you to decide / evaluate | 20:05 |
| lkcl | lqarx, that's what i wanted to search for | 20:05 |
| * lkcl ngggh stressed. 128-bit operations with atomic behavoiur, streeeeess | 20:06 | |
| lkcl | we need to check what Microwatt does, there | 20:07 |
| lkcl | kylel, ok so you've done python, and c, and assembly-level programming | 20:08 |
| lkcl | this is goood. | 20:08 |
| lkcl | would you be happy to go through this | 20:08 |
| lkcl | https://libre-soc.org/docs/firststeps/ | 20:08 |
| lkcl | which ghostmansd-pc wrote a couple weeks ago? | 20:08 |
| lkcl | it will give you an idea of how things "fit together" on the simulator side | 20:09 |
| ghostmansd-pc | yep, that's mine :-) | 20:09 |
| lkcl | :) | 20:09 |
| ghostmansd-pc | all faults are mine | 20:09 |
| kylel | lol | 20:09 |
| programmerjake | the lq and stq instructions are needed for compiling 128-bit atomic relaxed ld/st | 20:10 |
| ghostmansd-pc | interesting, do we have a proper map of how C11 atomics interoperate with openpower? | 20:10 |
| kylel | lkcl will do | 20:10 |
| programmerjake | since they're cheaper than a lqarx stqcx. loop | 20:10 |
| * lkcl has no clue how an elwidth ovrride would even work on lq / stq | 20:10 | |
| programmerjake | it won't... | 20:11 |
| programmerjake | it's like asking how elwidth override works on a syscall | 20:11 |
| lkcl | kylel, it will give you some confidence in the idea of a specification actually coming from an official source (OpenPOWER ISA v3.0B document) | 20:11 |
| lkcl | and *compiling* it into an actual simulator | 20:11 |
| lkcl | and some confidence in the fact that "if i modify the pseudocode, it changes the actual simulator... which... errr... isn't in python? moo?" | 20:12 |
| lkcl | there's a *python compiler* (!) which turns the v3.0B specification source code *INTO* python | 20:13 |
| lkcl | and also the actual Power ISA *decoder* is in plain ASCII text file format (again, extracted from the spec, into machine-readable format) | 20:13 |
| * lkcl deep breath | 20:13 | |
| ghostmansd-pc | kylel: despite how it looks, that's actually quite a nice stuff :-) | 20:13 |
| ghostmansd-pc | I really like the idea | 20:14 |
| lkcl | yeah, it means we can do a c compiler and get a c code simulator "for free" | 20:14 |
| lkcl | and there are people actively considering doing a Formal Proof (SAIL) as well | 20:14 |
| kylel | I was impressed by the pseudocode in the v3 manual | 20:14 |
| lkcl | doing a simple compiler | 20:14 |
| lkcl | yeah, it's pretty readable and consistent | 20:15 |
| lkcl | and it was an obvious natural step (at least to me) to ignore the warnings in the preamble, "warning, this is not intended to be executed", *make* it executable | 20:15 |
| lkcl | :) | 20:15 |
| lkcl | that involved adding ">u" and "<u" comparison operators (unsigned-less-than, unsigned-greater-than) | 20:16 |
| lkcl | and | 20:16 |
| lkcl | (sigh) | 20:16 |
| ghostmansd-pc | (actually it has some other differences, e.g. xor operator) | 20:16 |
| ghostmansd-pc | IIRC we even had these documented somewhere | 20:16 |
| ghostmansd-pc | I for sure recall I saw it | 20:17 |
| lkcl | painstakingly going through the spec (with unit tests as backup) to identify those comparators which, in the spec *wording* (in english), said, "this is a signed comparison" | 20:17 |
| lkcl | ghostmansd-pc, it's more that in the FrameMaker Source they picked a non-ASCII (scientific) symbol | 20:17 |
| ghostmansd-pc | well, bitwise negation is not an easy to type as well :-) | 20:18 |
| programmerjake | lq demo: https://gcc.godbolt.org/z/T1neMqdff | 20:18 |
| lkcl | programmerjake, thx | 20:18 |
| lkcl | programmerjake, can you raise a bugreport about this because, sigh, it needs properly investigating. | 20:18 |
| ghostmansd-pc | lkcl: would it be sufficient for a lazy ass like me to execute this in src/? $(for each in ./*; do cd "$each"; git pull --rebase --recurse-submodules origin master; cd ..; done) | 20:19 |
| lkcl | i have noooo idea for example what to do with elwidth overrides on lq / stq and it needs some thought | 20:19 |
| lkcl | ghostmansd-pc, pffh, perfect | 20:19 |
| ghostmansd-pc | (I don't account for errors, perhaps I'll write a script) | 20:19 |
| lkcl | add it to dev-env-setup :) | 20:19 |
| lkcl | i just gave you write-perms :) | 20:20 |
| ghostmansd-pc | lol | 20:20 |
| * lkcl need to get up and walk about | 20:20 | |
| ghostmansd-pc | OK, will do after tests | 20:20 |
| programmerjake | i'd say lq/stq don't support elwidth override | 20:20 |
| lkcl | sitting too long | 20:20 |
| lkcl | programmerjake, i'd tend to agree - we just need to document the reasons why | 20:20 |
| programmerjake | use ld/std if you want <128-bits | 20:21 |
| lkcl | it's that it's involved in atomics, that makes it important to have | 20:21 |
| * lkcl afk need to move about | 20:21 | |
| ghostmansd-pc | programmerjake: so instructiomns less-than-128-bit are implicitly atomic? | 20:26 |
| programmerjake | yes, if the load/stores are correctly aligned and going to normal memory...they are "relaxed" atomic | 20:28 |
| ghostmansd-pc | nice! any way to ensure barriers? | 20:31 |
| programmerjake | yup...add lwsync or sync before or after | 20:33 |
| programmerjake | for some details, see https://www.cl.cam.ac.uk/~pes20/cpp/cpp0xmappings.html | 20:33 |
| programmerjake | well...just had fun discovering that thermal sensor #2 of 5 failed on my odroid xu4... | 20:35 |
| programmerjake | it was randomly shutting down | 20:35 |
| programmerjake | i watched the readings and it randomly said 45 C and -25 C | 20:36 |
| programmerjake | i'm assuming the shutdowns are caused by it randomly saying 130 C or similar | 20:36 |
| ghostmansd-pc | basically power also uses ll/sc, right? | 20:37 |
| programmerjake | i told linux to fake the sensor reading, always using 42 C | 20:38 |
| ghostmansd-pc | I mean, they name it differently, but the logic is the same | 20:38 |
| programmerjake | ghostmansd yes | 20:38 |
| EmanuelLoos[m] | Hello, have you ever considerd developing an open hardware 5G modem or do you know if such a project exists? | 20:38 |
| ghostmansd-pc | ah, the answer for everything! | 20:38 |
| programmerjake | the instructions are lqarx stqcx. and friends | 20:39 |
| programmerjake | Emanuel Loos idk, sounds interesting! | 20:39 |
| EmanuelLoos[m] | programmerjake: The reason I am asking is that there appears to be no 4G modem usable without non-free drivers. The only non-free drivers the Librem 5 still needs are those for the modem. | 20:41 |
| programmerjake | I'm aware of https://bellard.org/lte/ idk if it's open source though | 20:43 |
| EmanuelLoos[m] | programmerjake: Would this be useable in a smartphone? | 20:45 |
| programmerjake | probably... | 20:45 |
| EmanuelLoos[m] | programmerjake: Do you mean the software or the hardware? | 20:47 |
| programmerjake | iirc it's software there, it's for software-defined-radio 4g/5g basestations | 20:48 |
| programmerjake | idk what's open source if anything or everything | 20:49 |
| EmanuelLoos[m] | programmerjake: How good is a "software-defined-radio 4g/5g basestation" compared to a modem? | 20:51 |
| programmerjake | idk | 20:52 |
| programmerjake | it's the part that runs a celltower, not the part in a cellphone | 20:52 |
| EmanuelLoos[m] | Do you think NLNet would support developing a 5G modem too? | 20:53 |
| programmerjake | probably | 20:53 |
| ghostmansd-pc | lkcl: two good news! | 20:54 |
| ghostmansd-pc | 1) I've finally been able to execute at least one test on talos | 20:54 |
| programmerjake | yay! | 20:54 |
| ghostmansd-pc | 2) test_ldst_compunit.py runs fine on xlen branch | 20:55 |
| programmerjake | ooh! | 20:55 |
| ghostmansd-pc | before executing this one, I did $(git checkout xlen), please, confirm, this is sufficient :-D | 20:55 |
| programmerjake | thx for your hard work, ghostmansd and lkcl | 20:55 |
| ghostmansd-pc | ah, wait, I forgot to update pywriter | 20:56 |
| ghostmansd-pc | one moment, will re-check | 20:56 |
| ghostmansd-pc | programmerjake: thank you too for your help! | 20:56 |
| EmanuelLoos[m] | programmerjake: Than I think it would be really great if that could be archived! | 20:57 |
| programmerjake | :) | 20:58 |
| ghostmansd-pc | ah, failed | 21:00 |
| ghostmansd-pc | anyway, at least I'm able to debug now :-) | 21:01 |
| ghostmansd-pc | ok, 1 good news out of two, still good! | 21:01 |
| ghostmansd-pc | will continue tomorrow | 21:01 |
| lkcl | ghostmansd-pc, well done | 21:02 |
| lkcl | k | 21:03 |
| lkcl | EmanuelLoos[m], look up the pinephone LTE modem reverse-engineering | 21:03 |
| lkcl | yes that's the sort of thing nlnet has funded in the past (Haralde Welte's OpenBTS and reverse-engineering of Calypso chipsets) | 21:04 |
| ghostmansd-pc | I couldn't resist and checked what gone wrong in xlen branch | 21:07 |
| ghostmansd-pc | should be fine now, pushing changes into master | 21:07 |
| ghostmansd-pc | lkcl, could you, please, take a quick look to check whether we covered all fixedstore? | 21:08 |
| ghostmansd-pc | actions done: push xlen into remote; ssh talos; cd ~/src/openpower-isa; git fetch -a; git checkout xlen; git pull origin xlen --rebase; src/openpower/decoder/pseudo/pywriter.py; cd ~/src/soc; python3 ./src/soc/fu/compunits/test/test_ldst_compunit.py | 21:12 |
| lkcl | looks good, a couple of optimisations: | 21:14 |
| lkcl | pywriter noall fixedstore | 21:14 |
| lkcl | (because *you* know you only modified fixedstore.mdwn) | 21:14 |
| lkcl | this will save several minutes of your time | 21:14 |
| lkcl | pywriter because it's been installed into ~/.local/bin and that *should* be in the $PATH | 21:15 |
| * lkcl rerunning test_ldst_compunit.py now... | 21:16 | |
| lkcl | yaaa confirmed, all good | 21:16 |
| * lkcl rerunning test_issuer.py for about the 14th time in 3 days... | 21:16 | |
| EmanuelLoos[m] | <lkcl> "yes that's the sort of thing..." <- Is it being developed further into free software drivers or, even better, an open hardware modem or has it been discontinued? | 21:34 |
| lkcl | EmanuelLoos[m], this was about 8 years ago | 21:39 |
| lkcl | https://nlnet.nl/project/openbts-hw/ | 21:39 |
| lkcl | 10 | 21:39 |
| lkcl | https://nlnet.nl/project/sdr-phy/ | 21:39 |
| lkcl | https://nlnet.nl/project/osld/ | 21:40 |
| lkcl | https://nlnet.nl/project/4G_FOSS/ | 21:40 |
| lkcl | https://nlnet.nl/project/OpenWifi-80211n/ | 21:41 |
| lkcl | and many others - full list is here https://nlnet.nl/project/ | 21:41 |
| EmanuelLoos[m] | So are there any open hardware modems resulting from these projects? | 21:48 |
Generated by irclog2html.py 2.17.1 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!