| *** lxo <lxo!~lxo@gateway/tor-sasl/lxo> has quit IRC | 00:50 | |
| *** lxo <lxo!~lxo@gateway/tor-sasl/lxo> has joined #libre-soc | 00:51 | |
| *** yambo <yambo!~yambo@069-145-110-003.biz.spectrum.com> has quit IRC | 01:19 | |
| *** yambo <yambo!~yambo@069-145-110-003.biz.spectrum.com> has joined #libre-soc | 01:30 | |
| *** lxo <lxo!~lxo@gateway/tor-sasl/lxo> has quit IRC | 01:47 | |
| *** lxo <lxo!~lxo@gateway/tor-sasl/lxo> has joined #libre-soc | 01:47 | |
| *** lxo <lxo!~lxo@gateway/tor-sasl/lxo> has quit IRC | 07:22 | |
| *** lxo <lxo!~lxo@gateway/tor-sasl/lxo> has joined #libre-soc | 07:23 | |
| *** ghostmansd <ghostmansd!~ghostmans@broadband-109-173-83-100.ip.moscow.rt.ru> has joined #libre-soc | 07:29 | |
| *** ghostmansd <ghostmansd!~ghostmans@broadband-109-173-83-100.ip.moscow.rt.ru> has quit IRC | 07:37 | |
| *** ghostmansd <ghostmansd!~ghostmans@broadband-109-173-83-100.ip.moscow.rt.ru> has joined #libre-soc | 08:37 | |
| *** ghostmansd <ghostmansd!~ghostmans@broadband-109-173-83-100.ip.moscow.rt.ru> has quit IRC | 08:53 | |
| *** ghostmansd <ghostmansd!~ghostmans@broadband-109-173-83-100.ip.moscow.rt.ru> has joined #libre-soc | 09:19 | |
| *** ghostmansd <ghostmansd!~ghostmans@broadband-109-173-83-100.ip.moscow.rt.ru> has quit IRC | 09:51 | |
| *** ghostmansd <ghostmansd!~ghostmans@broadband-109-173-83-100.ip.moscow.rt.ru> has joined #libre-soc | 10:12 | |
| *** ghostmansd <ghostmansd!~ghostmans@broadband-109-173-83-100.ip.moscow.rt.ru> has quit IRC | 11:01 | |
| *** ghostmansd <ghostmansd!~ghostmans@broadband-109-173-83-100.ip.moscow.rt.ru> has joined #libre-soc | 11:12 | |
| *** octavius <octavius!~octavius@92.40.168.36.threembb.co.uk> has joined #libre-soc | 11:17 | |
| *** ghostmansd[m] <ghostmansd[m]!~ghostmans@broadband-109-173-83-100.ip.moscow.rt.ru> has quit IRC | 11:19 | |
| *** ghostmansd <ghostmansd!~ghostmans@broadband-109-173-83-100.ip.moscow.rt.ru> has quit IRC | 11:19 | |
| *** ghostmansd[m] <ghostmansd[m]!~ghostmans@176.59.174.94> has joined #libre-soc | 11:20 | |
| lkcl | ghostmansd[m], arse i just messed up the branch-checkouts. | 12:14 |
|---|---|---|
| lkcl | i might be able to fix it | 12:15 |
| lkcl | ok sorted | 12:16 |
| lkcl | and also rebased the new branch, extra322_ldst_idx | 12:18 |
| lkcl | https://git.libre-soc.org/?p=openpower-isa.git;a=shortlog;h=refs/heads/extra322_ldst_idx | 12:18 |
| *** octavius_ <octavius_!~octavius@92.40.169.186.threembb.co.uk> has joined #libre-soc | 14:03 | |
| *** octavius <octavius!~octavius@92.40.168.36.threembb.co.uk> has quit IRC | 14:06 | |
| *** octavius_ <octavius_!~octavius@92.40.169.186.threembb.co.uk> has quit IRC | 14:22 | |
| *** octavius_ <octavius_!~octavius@92.40.169.139.threembb.co.uk> has joined #libre-soc | 14:23 | |
| lkcl | ghostmansd[m], when you've a moment (ignore me for now i'm just typing) | 15:00 |
| lkcl | a couple of thoughts occurred to me | 15:00 |
| lkcl | 1. for visitor-walking, the priority at the moment is disassembly (decoding). redoing PowerDecoder for example | 15:02 |
| lkcl | 2. i have an idea - don't try *separate* visitor-walking, actually have a function-pattern which if we do it really carefully can be part of the base-class of the entire insnsdb set! | 15:03 |
| lkcl | and if we call the "start-header" function "__entry__" and the "end-header" function "__exit__", that *is* the very definition of a context-manager! | 15:04 |
| lkcl | and so, if every class gains a pattern-of-functions (start-children-end) *and says what its children are* then it's a total no-brainer | 15:07 |
| lkcl | first page google search "contextmanager visitor pattern" shows this https://pyomo.readthedocs.io/en/stable/_modules/pyomo/core/expr/visitor.html | 15:07 |
| lkcl | class StreamBasedExpressionVisitor(object) | 15:08 |
| lkcl | ahhhhh | 15:08 |
| lkcl | that's it | 15:09 |
| lkcl | programmerjake, ah ha! notice the child_idx in "data = acceptChildResult(self, node, data, child_result, child_idx)" ? | 15:10 |
| lkcl | that's what you were talking about - i didn't quite understand it at the time | 15:10 |
| lkcl | i think we can literally use StreamBasedExpressionVisitor *as it is* | 15:11 |
| lkcl | and "conform" with it. | 15:11 |
| *** ghostmansd[m] <ghostmansd[m]!~ghostmans@176.59.174.94> has quit IRC | 15:42 | |
| *** ghostmansd[m] <ghostmansd[m]!~ghostmans@broadband-109-173-83-100.ip.moscow.rt.ru> has joined #libre-soc | 15:44 | |
| *** ghostmansd <ghostmansd!~ghostmans@broadband-109-173-83-100.ip.moscow.rt.ru> has joined #libre-soc | 15:45 | |
| *** ghostmansd[hexch <ghostmansd[hexch!~ghostmans@broadband-109-173-83-100.ip.moscow.rt.ru> has joined #libre-soc | 15:52 | |
| *** ghostmansd <ghostmansd!~ghostmans@broadband-109-173-83-100.ip.moscow.rt.ru> has quit IRC | 15:54 | |
| *** ghostmansd[hexch <ghostmansd[hexch!~ghostmans@broadband-109-173-83-100.ip.moscow.rt.ru> has quit IRC | 16:11 | |
| *** ghostmansd[hexch <ghostmansd[hexch!~ghostmans@broadband-109-173-83-100.ip.moscow.rt.ru> has joined #libre-soc | 16:15 | |
| *** ghostmansd[hexch <ghostmansd[hexch!~ghostmans@broadband-109-173-83-100.ip.moscow.rt.ru> has quit IRC | 16:25 | |
| *** ghostmansd[hexch <ghostmansd[hexch!~ghostmans@broadband-109-173-83-100.ip.moscow.rt.ru> has joined #libre-soc | 16:31 | |
| *** ghostmansd[hexch <ghostmansd[hexch!~ghostmans@broadband-109-173-83-100.ip.moscow.rt.ru> has quit IRC | 16:36 | |
| *** ghostmansd[m] <ghostmansd[m]!~ghostmans@broadband-109-173-83-100.ip.moscow.rt.ru> has quit IRC | 16:52 | |
| *** ghostmansd[m] <ghostmansd[m]!~ghostmans@176.59.42.252> has joined #libre-soc | 16:53 | |
| *** ghostmansd[m] <ghostmansd[m]!~ghostmans@176.59.42.252> has quit IRC | 17:02 | |
| *** ghostmansd[m] <ghostmansd[m]!~ghostmans@46.39.229.42> has joined #libre-soc | 17:03 | |
| *** ghostmansd[m] <ghostmansd[m]!~ghostmans@46.39.229.42> has quit IRC | 17:08 | |
| *** ghostmansd[m] <ghostmansd[m]!~ghostmans@176.59.42.252> has joined #libre-soc | 17:10 | |
| *** ghostmansd[m] <ghostmansd[m]!~ghostmans@176.59.42.252> has quit IRC | 17:13 | |
| *** ghostmansd[m] <ghostmansd[m]!~ghostmans@46.39.229.55> has joined #libre-soc | 17:13 | |
| lkcl | urrr the visitor-walker i added in nmutil is coming back to me | 18:03 |
| *** ghostmansd[m] <ghostmansd[m]!~ghostmans@46.39.229.55> has quit IRC | 18:21 | |
| *** ghostmansd[m] <ghostmansd[m]!~ghostmans@176.59.55.126> has joined #libre-soc | 18:22 | |
| *** ghostmansd[m] <ghostmansd[m]!~ghostmans@176.59.55.126> has quit IRC | 18:37 | |
| *** ghostmansd[m] <ghostmansd[m]!~ghostmans@broadband-109-173-83-100.ip.moscow.rt.ru> has joined #libre-soc | 18:37 | |
| lkcl | programmerjake, apologies my laptop is getting so overloaded (swap-thrashing) that i had to disable utf-8 and fcvt unit tests. | 18:55 |
| lkcl | at 56 GB resident RAM when my laptop is 64 GB it put the loadavg up to 100 and made it unresponsive. | 18:56 |
| lkcl | i can't have that situation occur | 18:56 |
| lkcl | can you sort that out by drastically reducing the number of tests? | 18:56 |
| lkcl | aside from anything it will kill the CI machine by putting it permanently into thrashing | 18:57 |
| *** ghostmansd <ghostmansd!~ghostmans@broadband-109-173-83-100.ip.moscow.rt.ru> has joined #libre-soc | 19:35 | |
| markos_ | does it need to run everything in parallel? | 20:13 |
| lkcl | only for "speed" | 20:33 |
| lkcl | it takes about 12 minutes @ 8 processes | 20:33 |
| ghostmansd | A huge comment on visitors: https://bugs.libre-soc.org/show_bug.cgi?id=1094#c5 | 20:34 |
| lkcl | on my 4.8 ghz laptop | 20:34 |
| lkcl | ghostmansd, awesome | 20:34 |
| ghostmansd | Take your time, this is huge indeed. | 20:34 |
| ghostmansd | If you don't have time to dive into it as whole, please answer in chunks, that's perfectly normal. I tried to address several parts so that this can be convenient for all of you guys. | 20:35 |
| lkcl | will do quick replies i have to take my daughter out shopping | 20:35 |
| ghostmansd | These comments are not call for action, but my thoughts. Let's discuss. | 20:36 |
| lkcl | give me 20 mins? | 20:36 |
| ghostmansd | No rush, this is a serious topic. | 20:36 |
| ghostmansd | Take as many minutes you need. :-) | 20:36 |
| ghostmansd | Even hours... but not years. :-) | 20:36 |
| lkcl | yep everything you said makes sense | 20:37 |
| lkcl | bbl | 20:37 |
| lkcl | btw do look at nmutil visitor | 20:39 |
| lkcl | https://git.libre-soc.org/?p=nmutil.git;a=blob;f=src/nmutil/nmoperator.py;hb=HEAD | 20:40 |
| lkcl | the important function is "flatten" | 20:41 |
| lkcl | it's very specific/explicit, the type-calling "if isinstance(ai, Record)" | 20:41 |
| lkcl | bad hack | 20:42 |
| programmerjake | lkcl, we still need those tests, but i was thinking we could skip utf8 and fcvt tests completely unless `ENABLE_SLOW_TESTS` env var is set | 21:33 |
| programmerjake | also, if you want to not run out of ram, just set SILENCELOG='!*,default' | 21:35 |
| programmerjake | since the memory is being used by the *literally dozens of gigabytes* of log messages | 21:36 |
| programmerjake | imho we need to remove/comment-out log calls in selectableint and other places that are called a bunch but already known to work | 21:37 |
| programmerjake | the tests themselves likely use much less than 1GB ram per parallel testing process | 21:40 |
| programmerjake | as an example, my build server is currently running ci on 8 cores and is using 3.47GB ram on the whole system | 21:45 |
| programmerjake | because ci has SILENCELOG='!*,default' set | 21:46 |
| programmerjake | ok, it's at 97% complete running pytest and is now up to 3.64GB | 21:49 |
| programmerjake | another idea: have SILENCELOG disable `default` log messages by default, you need to pass SILENCELOG='!*' to show them all | 21:52 |
| programmerjake | or SILENCELOG=0 | 21:58 |
| *** octavius_ <octavius_!~octavius@92.40.169.139.threembb.co.uk> has quit IRC | 22:03 | |
| lkcl | programmerjake, no, i need to be able to run them | 23:12 |
| lkcl | that's inconvenient as i run them frequently and examine them | 23:14 |
| lkcl | bit-for-bit and look for particular keywords to trace the behaviour of ISACaller. | 23:14 |
| lkcl | > because ci has SILENCELOG='!*,default' set | 23:14 |
| lkcl | good, that's perfect *for ci* | 23:14 |
| lkcl | even if classified as "slow" i *have* to have them running, and running locally | 23:17 |
| lkcl | and if "slow" and they cause *my* machine to run out of *64* GB of RAM that's not going to work, is it? | 23:18 |
| lkcl | as the person who has to keep an eye on everything | 23:18 |
| lkcl | (and whose much smaller laptop only has 16 GB of RAM) | 23:18 |
| lkcl | we *cannot* make the assumption that everyone (including me) has massive-resourced-machines at their disposal | 23:19 |
| programmerjake | well, then run all tests with SILENCELOG and if some fail, rerun *just those failing tests* without SILENCELOG so you can analyze their output | 23:27 |
| programmerjake | you can easily do that with pytest --lf which runs only the tests that failed last time | 23:30 |
| programmerjake | the way i see it, this problem is ultimately caused by our having too many log calls in places where we no longer need them, such as selectableint's methods | 23:32 |
| lkcl | i do it the other way round | 23:36 |
| lkcl | i *need* those tests to be able to run, ok? | 23:36 |
| lkcl | and i need them to run with full log files, *please listen* | 23:37 |
| programmerjake | oh, i have an idea, maybe we can make a simple pytest plugin that stores logs on disk instead of in ram... | 23:37 |
| lkcl | ah that would work great | 23:37 |
| programmerjake | or maybe just fix the pytest bug for that that's been open for a while iirc, lemme find it... | 23:39 |
| lkcl | ahhh https://github.com/pytest-dev/pytest/issues/2968#issuecomment-348204100 | 23:51 |
| lkcl | tearDown needs to explicitly close objects | 23:51 |
| programmerjake | i think i'll just write a plugin that captures output to a file for each test and gives the filename and first and last 5k lines back to pytest, if you want the rest you'll just have to look in the relevant file | 23:51 |
| programmerjake | the bug is that pytest passes the whole captured output in memory to the main process even for passed tests where it'll just ignore the output, so large outputs cause the main process to run out of memory | 23:54 |
| programmerjake | iirc | 23:54 |
| lkcl | yoww | 23:55 |
| programmerjake | (not the bug you linked which is already fixed) | 23:55 |
| lkcl | that sounds like a good idea | 23:55 |
| programmerjake | the store in files idea? | 23:56 |
| programmerjake | with just giving back first/last few lines? | 23:57 |
| lkcl | yes love it | 23:58 |
| programmerjake | ok, i'll work on that today | 23:59 |
| programmerjake | maybe can get some project management budget and publish it on pip | 23:59 |
Generated by irclog2html.py 2.17.1 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!