| mikolajw | ok, I have patched _pyrtl.py to output code that looks like C. The next step will be getting gcc to compile it and then somehow get pysim.py to step through it | 10:51 |
|---|---|---|
| mikolajw | I'll also need to somehow send Python variables to a C executable and then back, I don't have any plan for this yet | 10:56 |
| mikolajw | ah, but of course, via a foreign function interface | 10:58 |
| mikolajw | is Python CFFI allowed? | 11:00 |
| mikolajw | https://cffi.readthedocs.io/en/latest/ | 11:02 |
| lkcl | mikolajw hi, did you see my replies in IRC yesterday? | 11:03 |
| lkcl | and also on the bugtracker? | 11:03 |
| lkcl | unfortunately, programmerjake has given you wrong advice that will have led you down an unnecessary path, doing far more work than is required | 11:03 |
| mikolajw | yes, I did read what you wrote to me | 11:04 |
| lkcl | ok great. | 11:04 |
| * lkcl checking that i've given you access to openpower-isa | 11:04 | |
| lkcl | please do "git push" when you have completed any work within under 3-5 minutes of completing it or running any unit tests | 11:05 |
| lkcl | mikolajw, done. | 11:06 |
| lkcl | you should now have git commit write access to openpower-isa, please do "git push" to it | 11:07 |
| mikolajw | ok, just wait a sec, do you want me to have any unit tests for it already? It's going to break things if I push it now | 11:07 |
| lkcl | ok, so if that's the case, then there's something wrong. | 11:08 |
| mikolajw | well, it's just incomplete | 11:08 |
| lkcl | the code should have been written in such a way that it is totally stand-alone and does absolutely no damage whatsoever of any kind to existing code | 11:08 |
| lkcl | ah that's different, and perfectly fine | 11:08 |
| mikolajw | I mean, I've hardcoded the test to generate C (but it's not working yet) | 11:09 |
| lkcl | it's a common.... it's hard to describe, i don't quite know the right word even in english... | 11:09 |
| mikolajw | do you want this to be through some env var | 11:09 |
| mikolajw | so that you don't have to figure out how to run the working python tests | 11:10 |
| lkcl | it's a common... feeling / paradigm / something for people to go, "this code isn't working 100% therefore i must prevent and prohibit anyone from seeing it until it's perfect" | 11:10 |
| lkcl | which then means that you're the only person who can see what you're doing | 11:10 |
| lkcl | and nobody can do any code-review | 11:10 |
| lkcl | so, quick question: | 11:11 |
| lkcl | if you were to commit the code that you have written now, would it break any *existing* code? | 11:11 |
| lkcl | would it stop any unit tests written by anyone else from working, or stop any other code from working? | 11:11 |
| lkcl | if the answer to that is "no it wouldn't", then it's absolutely fine for you to commit what you have - right now - so that everyone can help out and review it. | 11:12 |
| lkcl | even if its own unit tests do not pass, that is perfectly fine. | 11:13 |
| lkcl | that's what unit tests are for: to show you (and everyone else) the progress, to act as a discussion point. | 11:13 |
| lkcl | what needs fixing, etc. etc. etc. | 11:13 |
| lkcl | but if those are not committed, how can we review it, help you, check that you're on the right track, etc. etc. etc.? | 11:14 |
| mikolajw | I'm sleepy now, so I'll just push it, and you will be able to adjust it if you won't like it | 11:14 |
| lkcl | ok :) | 11:15 |
| lkcl | i'll go over it in the bugtracker so you can take a look when you're ready. i've a lot else to do so will make some notes | 11:16 |
| mikolajw | lkcl: this was just a question of organization, some organizations for example require to create new patches, or patches by new contributors, on be on a separate branch so that they are reviewed before they are merged to master | 11:16 |
| lkcl | yes. i tend to throw people in at the deep end and trust them :) | 11:16 |
| mikolajw | i just need to be sure what are your requirements (i did read the tutorial on the website, yes) | 11:17 |
| mikolajw | ok, i'll just need to copy my SSH key to schroot environment | 11:17 |
| lkcl | if it makes you more comfortable to use a temporary branch, and you're happy with doing merges (yourself) then that's fine | 11:17 |
| lkcl | working in master branch makes sure that people communicate. | 11:18 |
| lkcl | i've worked on too many projects where months and in one case several *years* of work was discarded because it was "in a branch" | 11:18 |
| mikolajw | ok, push didn't work, "fatal: unable to access 'https://git.libre-soc.org/git/openpower-isa.git/': The requested URL returned error: 403" | 11:31 |
| mikolajw | probably some trivial error from my side | 11:33 |
| lkcl | ah you need the gitolite3 URL | 11:35 |
| lkcl | HTTPS is read-only for public access | 11:35 |
| lkcl | so it will be... | 11:35 |
| lkcl | 1 sec... | 11:35 |
| lkcl | [remote "origin"] | 11:35 |
| lkcl | url = gitolite3@git.libre-soc.org:openpower-isa.git | 11:35 |
| lkcl | you can hand-edit the src/openpower-isa/.git/config file to add that url | 11:35 |
| lkcl | as a replacement for url = https://..... | 11:36 |
| lkcl | i am _not_ setting up web server authentication on the server, it would become a really nice attack vector for the many bots that hammer it :) | 11:37 |
| mikolajw | >ssh: connect to host git.libre-soc.org port 22: Network is unreachable | 11:39 |
| mikolajw | >fatal: Could not read from remote repository. | 11:39 |
| mikolajw | >Please make sure you have the correct access rights | 11:39 |
| mikolajw | and the repository exists. | 11:39 |
| lkcl | ah port 922 | 11:39 |
| lkcl | edit ~/.ssh/config and add these lines | 11:39 |
| lkcl | Host git.libre-soc.org | 11:39 |
| lkcl | Port 922 | 11:40 |
| mikolajw | aw | 11:40 |
| lkcl | there's another way to do it, i think it's on the HDL_workflow page | 11:40 |
| mikolajw | ok, that one was in the tutorial | 11:40 |
| lkcl | yeah i moved to port 922 due to the insanely high number of port-scans | 11:40 |
| mikolajw | probably I had resetted my .ssh/config some time ago accidentally | 11:41 |
| lkcl | thousands of login attempts per day to random usernames | 11:41 |
| lkcl | ohh i bet you may have Host libre-soc.org Port 922 | 11:41 |
| lkcl | i made that mistake | 11:42 |
| lkcl | it's the same IP address (for now) | 11:42 |
| mikolajw | >FATAL: W any openpower-isa mikolaj DENIED by fallthru | 11:42 |
| lkcl | but ssh treats them as different hosts | 11:42 |
| lkcl | doh, my fault | 11:43 |
| lkcl | try again | 11:43 |
| lkcl | i put the name "mikolajw" in the write-perms not a key named "mikolaj" | 11:43 |
| mikolajw | pushed | 11:44 |
| lkcl | hooraaay :) | 11:44 |
| lkcl | ok will take a look later | 11:44 |
| lkcl | thank you mikolajw | 11:44 |
| mikolajw | you need to uncomment a line to enable conversion to C so that the Python simulation doesn't get borked | 11:45 |
| mikolajw | and I renew my question, can I use CFFI to call C from Python, or is there a different plan (did I miss something again?) | 11:46 |
| lkcl | ok - that's why i said to take a copy of any bits of code needed | 11:46 |
| lkcl | mmm.... i'm really not a fan of cffi | 11:46 |
| lkcl | and portability is not important | 11:46 |
| mikolajw | I don't understand what the phrase "take a copy of any bits of code needed" means | 11:47 |
| lkcl | (portability to e.g. pypy or windows etc. i mean) | 11:47 |
| lkcl | rather than monkey-patching | 11:47 |
| mikolajw | it's not monkey-patched now | 11:47 |
| lkcl | ahh wheww :) | 11:47 |
| mikolajw | i don't want to use CFFI for portability, I just expect it to be easier | 11:49 |
| mikolajw | I'll probably be able to call C straight from Python, as it's modeled after LuaJIT's FFI, which I remember faintly | 11:50 |
| lkcl | i found it surprisingly easy to output python c-based modules | 11:50 |
| mikolajw | with what? | 11:50 |
| lkcl | python print statements (pretty much exactly the same as in cxxrtl and pysim itself) | 11:50 |
| mikolajw | I'm probably missing something, but I don't understand what you mean | 11:51 |
| lkcl | it was for python-webkit. 350 python c-based modules auto-generated | 11:51 |
| mikolajw | you mean generating stuff that uses this?: https://docs.python.org/3/extending/extending.html | 11:52 |
| lkcl | so the library linking to webkit DOM was auto-generated | 11:52 |
| lkcl | yes. | 11:53 |
| lkcl | that's the one. | 11:53 |
| mikolajw | ok, so CFFI is a no | 11:53 |
| mikolajw | (or is it?) | 11:53 |
| lkcl | well, i don't know yet, i haven't looked at it. | 11:53 |
| lkcl | Jean-Paul Chaput (the author of coriolis2) considers these things to be so alarmingly flakey that he wrote his own | 11:54 |
| mikolajw | well, I had good experiences with LuaJIT's FFI, and they say they're modeling after it, so I'm inclined towards it | 11:54 |
| lkcl | and he's the author of a high-performance VLSI codebase written in mixed c++ and python | 11:54 |
| lkcl | ctypes is definitely out | 11:54 |
| lkcl | it constructs the stack manually (!!!) | 11:55 |
| lkcl | and there's no checking - at all: it's your responsibility to construct the function parameters on the stack correctly! | 11:55 |
| lkcl | i'll have a read of cffi | 11:56 |
| lkcl | to be honest if it's easier then pffh, why not | 11:56 |
| lkcl | okaaay, so cffi (if you follow this example) https://cffi.readthedocs.io/en/latest/overview.html#if-you-don-t-have-an-already-installed-c-library-to-call | 12:03 |
| lkcl | auto-generates some code _pi.c | 12:03 |
| lkcl | which contains the python-based c modules i was suggesting doing by hand | 12:04 |
| lkcl | basically it's a much more generalised version of the work i did for python-webkit | 12:04 |
| lkcl | looks great. | 12:04 |
| lkcl | love it | 12:04 |
| lkcl | static PyObject * | 12:06 |
| lkcl | _cffi_f_pi_approx(PyObject *self, PyObject *arg0) | 12:06 |
| lkcl | { | 12:06 |
| lkcl | float result; | 12:06 |
| lkcl | PyObject *pyresult; | 12:06 |
| lkcl | etc. etc. etc. | 12:06 |
| lkcl | perfect. it builds the c-based module which calls the function (in c) | 12:07 |
| lkcl | consequently you *can't* run into errors by mis-declaring the function header | 12:08 |
| lkcl | because the c compiler will throw a compile-time error for you | 12:08 |
| lkcl | whereas with ctypes it goes, "want that on the stack? let me just corrupt memory for you"... | 12:08 |
| lkcl | :) | 12:08 |
| lkcl | i did all that stuff - _pi.c - auto-generated in python-webkit. because of the regular nature and the small number of different types of data structures, it didn't take me long. | 12:10 |
| lkcl | i think i had something working in like 10 days flat, but it was my 2nd attempt: the first one was a webkit-gobject code-generator/compiler | 12:11 |
| lkcl | but this is great: ffibuilder.cdef("float pi_approx(int n);") | 12:11 |
| lkcl | a parser of c function headers | 12:12 |
| lkcl | https://www.youtube.com/watch?v=3qtUGzhEB_4 | 15:57 |
| octavius | lkcl, very fun! Please share when it comes, would be cool to hear the difference | 17:38 |
| tplaten | currently having a look at FetchFSM | 17:41 |
| tplaten | the big difference here is that microwatt does not have SVP64 | 17:44 |
| lkcl | tplaten: yes, the entirety of TestIssuer would only be around 450 lines (peanuts) without SVP64 | 18:34 |
| tplaten | TestIssuer is small, TestIssuerInternal is much larger. I vaguely remember that the unit tests only use TestIssuerInternal | 18:47 |
| tplaten | I now have a look at src/soc/simple/test/test_runner.py | 18:48 |
| lkcl | ah yes sorry, TestIssuerInternal, i keep caling them by the same name | 18:51 |
| lkcl | test_runner.py now uses a base class from openpower-isa | 18:51 |
| lkcl | https://git.libre-soc.org/?p=openpower-isa.git;a=blob;f=src/openpower/test/runner.py;hb=HEAD | 18:52 |
| lkcl | thanks to kyle for doing that | 18:52 |
| tplaten | there are two classes in test_runner.py HDLRunner and TestRunner, so two classes inherited | 18:54 |
| lkcl | ah not quite: HDLRunner is "registered" with the test-running system | 18:55 |
| lkcl | it's all slightly under development still | 18:55 |
| lkcl | so the idea is, there will be a QemuRunner, and a Gem5Runner, and a MicrowattRunner | 18:57 |
| tplaten | That looks good | 18:57 |
| tplaten | The MicrowattRunner would use the version of Microwatt tranlated to Verilog I guess, so libre-soc could be tested against microwatt | 18:58 |
| lkcl | correct. probably via... cocotb | 18:59 |
| tplaten | I had a look at OP_FETCH_FAILED, this performs a radix lookup, and if that fails too, an exception has to be reported back | 19:44 |
| lkcl | yes | 19:45 |
| lkcl | in a similar way to how exceptions work | 19:46 |
| lkcl | https://git.libre-soc.org/?p=openpower-isa.git;a=commitdiff;h=HEAD;hp=c270fe2b559fc421b4bc976aaf8a3cc5d72f8daa | 19:46 |
| lkcl | i'm currently wondering how to parse the tree in order to get at the I-Cache output data structure, icache.i_out, of type ICacheToDecode1Type | 19:47 |
| lkcl | that's what has the "fetch failed" signal | 19:47 |
| lkcl | which needs to be noted by TestIssuer[Internal] | 19:48 |
| lkcl | dropped into PowerDecoder2 | 19:48 |
| lkcl | PowerDecoder2 goes "oh an instruction fault occurred, let me just create a fake pseudo-op OP_FETCH_FAILED for that" | 19:48 |
| lkcl | that gets passed to the MMU FSM | 19:48 |
| lkcl | MMU FSM triggers LoadStore1 by raising ldst.instr_fault | 19:49 |
| lkcl | LoadStore1's FSM goes "oh look instr fault occurred, let's notify the MMU" | 19:49 |
| lkcl | MMU's FSM notices "iside" MMU_LOOKUP, does the RADIX walk and raises itlb_load (instruction TLB PTE load) | 19:50 |
| lkcl | itlb_load gets noticed by I-Cache and puts the PTE into its SRAMs | 19:51 |
| lkcl | (only at that point it drops its icache.i_out.instr_fault flag) | 19:51 |
| lkcl | TestIssuer notices that and drops the instr_fault over-ride on PowerDecoder2 | 19:52 |
| lkcl | then the fetch can be re-tried and this time it should succeed | 19:52 |
| lkcl | what you are referring to if the itlb_load fails, an exception would be raised | 19:53 |
| lkcl | we really need a unit test for that. | 19:53 |
| lkcl | test_loadstore1_ifetch_invalid() | 19:54 |
| tplaten | I could do that one | 19:58 |
| lkcl | that would be great. | 19:59 |
| lkcl | let me just add a call to get_exception_info, gimme 2mins | 20:02 |
| lkcl | done | 20:14 |
| tplaten | I continue tomorrow | 21:05 |
| *** tplaten <tplaten!~isengaara@55d4804c.access.ecotel.net> has left #libre-soc | 21:05 | |
| mikolajw | lkcl: just a question, was Cython previously considered to translate that Python simulation code to C? | 23:34 |
| mikolajw | I mean the generated Python code | 23:42 |
Generated by irclog2html.py 2.17.1 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!