Wednesday, 2021-12-08

mikolajwok, I have patched to output code that looks like C. The next step will be getting gcc to compile it and then somehow get to step through it10:51
mikolajwI'll also need to somehow send Python variables to a C executable and then back, I don't have any plan for this yet10:56
mikolajwah, but of course, via a foreign function interface10:58
mikolajwis Python CFFI allowed?11:00
lkclmikolajw hi, did you see my replies in IRC yesterday?11:03
lkcland also on the bugtracker?11:03
lkclunfortunately, programmerjake has given you wrong advice that will have led you down an unnecessary path, doing far more work than is required11:03
mikolajwyes, I did read what you wrote to me11:04
lkclok great.11:04
* lkcl checking that i've given you access to openpower-isa11:04
lkclplease do "git push" when you have completed any work within under 3-5 minutes of completing it or running any unit tests11:05
lkclmikolajw, done.11:06
lkclyou should now have git commit write access to openpower-isa, please do "git push" to it11:07
mikolajwok, just wait a sec, do you want me to have any unit tests for it already? It's going to break things if I push it now11:07
lkclok, so if that's the case, then there's something wrong.11:08
mikolajwwell, it's just incomplete11:08
lkclthe code should have been written in such a way that it is totally stand-alone and does absolutely no damage whatsoever of any kind to existing code11:08
lkclah that's different, and perfectly fine11:08
mikolajwI mean, I've hardcoded the test to generate C (but it's not working yet)11:09
lkclit's a common.... it's hard to describe, i don't quite know the right word even in english...11:09
mikolajwdo you want this to be through some env var11:09
mikolajwso that you don't have to figure out how to run the working python tests11:10
lkclit's a common... feeling / paradigm / something for people to go, "this code isn't working 100% therefore i must prevent and prohibit anyone from seeing it until it's perfect"11:10
lkclwhich then means that you're the only person who can see what you're doing11:10
lkcland nobody can do any code-review11:10
lkclso, quick question:11:11
lkclif you were to commit the code that you have written now, would it break any *existing* code?11:11
lkclwould it stop any unit tests written by anyone else from working, or stop any other code from working?11:11
lkclif the answer to that is "no it wouldn't", then it's absolutely fine for you to commit what you have - right now - so that everyone can help out and review it.11:12
lkcleven if its own unit tests do not pass, that is perfectly fine.11:13
lkclthat's what unit tests are for: to show you (and everyone else) the progress, to act as a discussion point.11:13
lkclwhat needs fixing, etc. etc. etc.11:13
lkclbut if those are not committed, how can we review it, help you, check that you're on the right track, etc. etc. etc.?11:14
mikolajwI'm sleepy now, so I'll just push it, and you will be able to adjust it if you won't like it11:14
lkclok :)11:15
lkcli'll go over it in the bugtracker so you can take a look when you're ready.  i've a lot else to do so will make some notes11:16
mikolajwlkcl: this was just a question of organization, some organizations for example require to create new patches, or patches by new contributors, on be on a separate branch so that they are reviewed before they are merged to master11:16
lkclyes.  i tend to throw people in at the deep end and trust them :)11:16
mikolajwi just need to be sure what are your requirements (i did read the tutorial on the website, yes)11:17
mikolajwok, i'll just need to copy my SSH key to schroot environment11:17
lkclif it makes you more comfortable to use a temporary branch, and you're happy with doing merges (yourself) then that's fine11:17
lkclworking in master branch makes sure that people communicate.11:18
lkcli've worked on too many projects where months and in one case several *years* of work was discarded because it was "in a branch"11:18
mikolajwok, push didn't work, "fatal: unable to access '': The requested URL returned error: 403"11:31
mikolajwprobably some trivial error from my side11:33
lkclah you need the gitolite3 URL11:35
lkclHTTPS is read-only for public access11:35
lkclso it will be...11:35
lkcl1 sec...11:35
lkcl[remote "origin"]11:35
lkcl    url =
lkclyou can hand-edit the src/openpower-isa/.git/config file to add that url11:35
lkclas a replacement for url = https://.....11:36
lkcli am _not_ setting up web server authentication on the server, it would become a really nice attack vector for the many bots that hammer it :)11:37
mikolajw>ssh: connect to host port 22: Network is unreachable11:39
mikolajw>fatal: Could not read from remote repository.11:39
mikolajw>Please make sure you have the correct access rights11:39
mikolajwand the repository exists.11:39
lkclah port 92211:39
lkcledit ~/.ssh/config and add these lines11:39
lkclHost git.libre-soc.org11:39
lkcl    Port 92211:40
lkclthere's another way to do it, i think it's on the HDL_workflow page11:40
mikolajwok, that one was in the tutorial11:40
lkclyeah i moved to port 922 due to the insanely high number of port-scans11:40
mikolajwprobably I had resetted my .ssh/config some time ago accidentally11:41
lkclthousands of login attempts per day to random usernames11:41
lkclohh i bet you may have Host Port 92211:41
lkcli made that mistake11:42
lkclit's the same IP address (for now)11:42
mikolajw>FATAL: W any openpower-isa mikolaj DENIED by fallthru11:42
lkclbut ssh treats them as different hosts11:42
lkcldoh, my fault11:43
lkcltry again11:43
lkcli put the name "mikolajw" in the write-perms not a key named "mikolaj"11:43
lkclhooraaay :)11:44
lkclok will take a look later11:44
lkclthank you mikolajw11:44
mikolajwyou need to uncomment a line to enable conversion to C so that the Python simulation doesn't get borked11:45
mikolajwand I renew my question, can I use CFFI to call C from Python, or is there a different plan (did I miss something again?)11:46
lkclok - that's why i said to take a copy of any bits of code needed11:46
lkclmmm.... i'm really not a fan of cffi11:46
lkcland portability is not important11:46
mikolajwI don't understand what the phrase "take a copy of any bits of code needed" means11:47
lkcl(portability to e.g. pypy or windows etc. i mean)11:47
lkclrather than monkey-patching11:47
mikolajwit's not monkey-patched now11:47
lkclahh wheww :)11:47
mikolajwi don't want to use CFFI for portability, I just expect it to be easier11:49
mikolajwI'll probably be able to call C straight from Python, as it's modeled after LuaJIT's FFI, which I remember faintly11:50
lkcli found it surprisingly easy to output python c-based modules11:50
mikolajwwith what?11:50
lkclpython print statements (pretty much exactly the same as in cxxrtl and pysim itself)11:50
mikolajwI'm probably missing something, but I don't understand what you mean11:51
lkclit was for python-webkit.  350 python c-based modules auto-generated11:51
mikolajwyou mean generating stuff that uses this?:
lkclso the library linking to webkit DOM was auto-generated11:52
lkclthat's the one.11:53
mikolajwok, so CFFI is a no11:53
mikolajw(or is it?)11:53
lkclwell, i don't know yet, i haven't looked at it.11:53
lkclJean-Paul Chaput (the author of coriolis2) considers these things to be so alarmingly flakey that he wrote his own11:54
mikolajwwell, I had good experiences with LuaJIT's FFI, and they say they're modeling after it, so I'm inclined towards it11:54
lkcland he's the author of a high-performance VLSI codebase written in mixed c++ and python11:54
lkclctypes is definitely out11:54
lkclit constructs the stack manually (!!!)11:55
lkcland there's no checking - at all: it's your responsibility to construct the function parameters on the stack correctly!11:55
lkcli'll have a read of cffi11:56
lkclto be honest if it's easier then pffh, why not11:56
lkclokaaay, so cffi (if you follow this example)
lkclauto-generates some code _pi.c12:03
lkclwhich contains the python-based c modules i was suggesting doing by hand12:04
lkclbasically it's a much more generalised version of the work i did for python-webkit12:04
lkcllooks great.12:04
lkcllove it12:04
lkclstatic PyObject *12:06
lkcl_cffi_f_pi_approx(PyObject *self, PyObject *arg0)12:06
lkcl  float result;12:06
lkcl  PyObject *pyresult;12:06
lkcletc. etc. etc.12:06
lkclperfect.  it builds the c-based module which calls the function (in c)12:07
lkclconsequently you *can't* run into errors by mis-declaring the function header12:08
lkclbecause the c compiler will throw a compile-time error for you12:08
lkclwhereas with ctypes it goes, "want that on the stack? let me just corrupt memory for you"...12:08
lkcli did all that stuff - _pi.c - auto-generated in python-webkit.  because of the regular nature and the small number of different types of data structures, it didn't take me long.12:10
lkcli think i had something working in like 10 days flat, but it was my 2nd attempt: the first one was a webkit-gobject code-generator/compiler12:11
lkclbut this is great: ffibuilder.cdef("float pi_approx(int n);")12:11
lkcla parser of c function headers12:12
octaviuslkcl, very fun! Please share when it comes, would be cool to hear the difference17:38
tplatencurrently having a look at FetchFSM17:41
tplatenthe big difference here is that microwatt does not have SVP6417:44
lkcltplaten: yes, the entirety of TestIssuer would only be around 450 lines (peanuts) without SVP6418:34
tplatenTestIssuer is small, TestIssuerInternal is much larger. I vaguely remember that the unit tests only use TestIssuerInternal18:47
tplatenI now have a look at src/soc/simple/test/test_runner.py18:48
lkclah yes sorry, TestIssuerInternal, i keep caling them by the same name18:51 now uses a base class from openpower-isa18:51
lkclthanks to kyle for doing that18:52
tplatenthere are two classes in HDLRunner and TestRunner, so two classes inherited18:54
lkclah not quite: HDLRunner is "registered" with the test-running system18:55
lkclit's all slightly under development still18:55
lkclso the idea is, there will be a QemuRunner, and a Gem5Runner, and a MicrowattRunner18:57
tplatenThat looks good18:57
tplatenThe MicrowattRunner would use the version of Microwatt tranlated to Verilog I guess, so libre-soc could be tested against microwatt18:58
lkclcorrect.  probably via... cocotb18:59
tplatenI had a look at OP_FETCH_FAILED, this performs a radix lookup, and if that fails too, an exception has to be reported back19:44
lkclin a similar way to how exceptions work19:46
lkcli'm currently wondering how to parse the tree in order to get at the I-Cache output data structure, icache.i_out, of type ICacheToDecode1Type19:47
lkclthat's what has the "fetch failed" signal19:47
lkclwhich needs to be noted by TestIssuer[Internal]19:48
lkcldropped into PowerDecoder219:48
lkclPowerDecoder2 goes "oh an instruction fault occurred, let me just create a fake pseudo-op OP_FETCH_FAILED for that"19:48
lkclthat gets passed to the MMU FSM19:48
lkclMMU FSM triggers LoadStore1 by raising ldst.instr_fault19:49
lkclLoadStore1's FSM goes "oh look instr fault occurred, let's notify the MMU"19:49
lkclMMU's FSM notices "iside" MMU_LOOKUP, does the RADIX walk and raises itlb_load (instruction TLB PTE load)19:50
lkclitlb_load gets noticed by I-Cache and puts the PTE into its SRAMs19:51
lkcl(only at that point it drops its icache.i_out.instr_fault flag)19:51
lkclTestIssuer notices that and drops the instr_fault over-ride on PowerDecoder219:52
lkclthen the fetch can be re-tried and this time it should succeed19:52
lkclwhat you are referring to if the itlb_load fails, an exception would be raised19:53
lkclwe really need a unit test for that.19:53
tplatenI could do that one19:58
lkclthat would be great.19:59
lkcllet me just add a call to get_exception_info, gimme 2mins20:02
tplatenI continue tomorrow21:05
*** tplaten <tplaten!> has left #libre-soc21:05
mikolajwlkcl: just a question, was Cython previously considered to translate that Python simulation code to C?23:34
mikolajwI mean the generated Python code23:42

Generated by 2.17.1 by Marius Gedminas - find it at!