Thursday, 2021-08-12

lkclghostmansd, no, use multiple instructions per test: see latest commits.02:27
lkcli saved some time by creating one (shared) PowerDecoder.02:27
lkclone other way to get even more instructions per test is to use LD/STs02:28
lkcleven write a small program that does a loop counter, writes to memory.02:28
lkclbut let's see how just hitting 32 bcd instructions at a time works, first02:29
justinrestivo[m]lkcl:  I am trying to build the various libre-soc repos on ubuntu. To install all the dependencies/build the libre-soc repos, should I just be sequentially running all the scripts here: https://git.libre-soc.org/?p=dev-env-setup.git? Or should I word-for-word be following the script on 1.2 here: https://libre-soc.org/docs/?02:31
lkclno use the instructions at https://libre-soc.org/HDL_workflow/devscripts/03:55
lkclyou need the base dependencies first03:55
lkcl./install-hdl-apt-reqs03:56
lkclinspect the contents, it should be reasonably self-evident03:56
lkclone thing: you *must not* let pip install sub-dependencies03:58
lkclpip is a f****g nuisance.03:58
lkclit goes, "oh you want nmigen-boards? let me go get a completely random version of nmigen for you, and screw you over"03:58
lkclto stop that the hdl repos *have* to be done in the order listed in ./hdl-dev-repos03:59
programmerjakefound this nice article on why Kraken is switching from PHP (and other stuff) to Rust06:07
programmerjakehttps://blog.kraken.com/post/7964/oxidizing-kraken-improving-kraken-infrastructure-using-rust/06:07
justinrestivo[m]Ah that's clearer. Will give it a shot, thanks.10:49
lkclprogrammerjake, this is the Libre-SOC Project, we don't use PHP (or kraken)13:31
lkclcan we please keep this channel focussed on issues related to Libre-SOC?13:32
programmerjakeyeah, but it does explain a lot of why I've been advocating for Libre-SOC to use Rust more, so I'd consider it on-topic13:33
programmerjakeI probably should have stated that in my original message13:35
lkclwhen speaking with the team in India, none of them knew of nmigen however all of them knew of python and had used it13:35
lkcli will say it again - in stronger terms this time - it would be really, really stupid of us to use rust as the primary language for Libre-SOC HDL13:36
lkclwe would be LITERALLY the only users of the HDL13:36
lkclwe would have serious training and recruitment problems13:36
lkcland it would severely jeapordise our chances of success13:36
lkclrust is NOT an appropriate lanugage as a critical hard dependency for a project that is this complex13:37
lkclwe are already in serious need of contributors13:37
lkclto force every single person to have to learn a language that is NOT WIDELY ADOPTED forces us to train them and pay them for that training time13:38
lkclbecause python is known by 30% of the world's programmers that cost is ELIMINATED13:38
lkclsorry to have to keep repeating this, jacob, please understand and accept it13:38
lkclwe went through this right at the beginning of the project, in the review and choice of the HDL13:39
lkclwe picked nmigen because it has a vibrant pre-existing community and its primary developer is extremely smart and committed13:40
lkclif we developed a rust-based HDL we would first have to develop that HDL and we would set the entire project back by at least another YEAR.13:40
lkclthen because we were literally the only team that would be using it we would be completely incapable of finding developers13:41
lkclwe went through this13:41
lkclso pleae13:41
lkclplease don't keep advocating rust all the time13:41
programmerjakemaybe not using Rust for the HDL, but other surrounding software?13:41
lkclthe exact same logic applies.13:41
lkclwe need to augment gcc.13:42
lkclgcc is not written in rust13:42
lkclwe need to augment binutils13:42
lkclbinutils is not written in rust13:42
lkclwe need to augment llvm13:42
lkclllvm is not written in rust13:42
lkclwe need to augment u-boot and coreboot13:42
lkclu-boot and coreboot are not written in rust13:42
lkclwe need to port libc6 to SVP6413:42
lkcllibc6 is not written in rust13:42
lkclevery single *immediate* focus - everything on the critical path - that we have to do and have funding for - is *not* written in rust13:43
lkcli've said that a number of times as well13:43
lkcli appreciate fully that you love rust: the danger is that it's becoming an obsession for you.13:44
lkcli've had that happen, myself: it's how i got so much done.13:45
lkclbut13:45
programmerjakeI'm not saying rewrite binutils/etc. in Rust, I'm saying new software can be written in Rust -- because the software doesn't exist or is of such quality that it needs rewriting -- NOT because the software just happens to be not in Rust13:45
lkcli also had to be told, when i was around your age, by managers at companies that i worked for, that i needed to focus on the tasks that were in front of me13:45
lkclthen the same things apply13:46
lkclif we do that, we have to pay people to train them in rust13:46
lkcland because it's not a commonly well-known language, we place the project at risk by choosing that language13:46
lkclyou're the *only* person in the entire world who knows pia, for example.13:47
lkclfinding someone to help work on it is seriously problematic for us13:47
lkclby contrast if you pick literally any random programmer in the world, there's a 1-in-3 chance that they'll know python13:48
lkclthis is what i've been trying to get across to you for a long, long time:13:48
lkclit's *not* just about how good the language is13:48
lkclwe have to think, "how can we find resources and a supportive ecosystem"13:49
programmerjakehmm, cole might know pia...also 1 in 12 or so programmers know Rust so it's not as if no one knows it...13:49
lkclpicking a language that is only known by a small percentage of the world's population is extremely unwise13:49
lkcl1 in 1213:49
lkclvs 1 in 313:49
lkcl*below* ten percent.13:50
lkclit's far too great a risk13:50
lkclplus, when analysing available projects that might do a similar job (and could be adapated), there's *just not enough available* yet.13:51
lkclpart of the applications to NLnet required doing a comparative analysis13:51
programmerjakealso, I went out of my way to add Python-visible docs to pia13:51
lkcl"please give a list of similar projects and compare them"13:52
lkclbasically they wanted to know if the money was being wasted on duplicating existing effort13:52
lkcli know you did - that's missing the point though13:52
lkclthis is basic project management13:53
lkclthe capabilities of a given language itself is a *minor* factor13:53
lkcladoption13:53
lkclmindshare13:54
lkclmaturity13:54
lkclstability13:54
lkclthese are FAR more important13:54
lkcli'm reading that article, i get why they chose rust.  they're security-minded, they have to protect users.13:55
lkclit's a good choice13:56
lkclbecause they are developing a networking product in software13:56
lkclLibre-SOC's development has *nothing to do with networking*13:56
programmerjakeok. so the adoption ... etc. is more of a factor than you just don't want to learn rust now13:56
lkclwe're not developing a networking product13:57
lkclit's both.13:57
lkcli have to focus on the immediate critical tasks, none of which require rust in any way13:57
programmerjakeummm...we got funding for a router ... how's that not networking?13:57
lkcli also have to think "how can i find other people and encourage them to participate"13:57
lkclthere's a firebreak between using a language because it has secure features13:58
lkclwhich protects against networking errors at the software level13:58
lkcland13:58
lkclwe have some HDL that has to be be expressed somehow13:58
jntoolchain components won't run on the hardware that will use the Libre-SoC;  in that sense, the toolchain components aren't network software13:59
lkclthere's a cognitive break between the two13:59
lkclbasically, yes.13:59
lkclonce we have a processor that EXECUTES rust programs, then there is security involved in the DEVELOPMENT of those programs14:00
programmerjakebut we will be writing networking software...a router is fine and good but is just an expensive heater without software14:00
lkclhowever that has ABSOLUTELY NOTHING to do with the tools USED to develop the hardware THAT executes those end-user programs14:00
jnnot all software that is somehow related to LS might be classified as "toolchain", but then we get into details rather than general tendencies14:00
lkclyeeees...14:00
lkclbut just as i said14:00
lkclthat software has *nothing to do with the tools used to develop that hardware*14:00
lkclhow can it?14:00
lkclthere is one example which helps illustrate this14:01
lkclif we were to develop some special custom hardware feature, such as Cambridge Capability14:01
lkclor memory pointer type-safety14:01
lkcl(in hardware)14:01
lkclthis btw was something that both lowRISC are researching, and IIT Madras as well, both in different ways14:01
programmerjakegood luck on compiler support!14:02
lkclTHEN we are focussing on hardware-assisted security14:02
lkclthis would... yes :)14:02
lkclthis would "tie in" to the actual software (and, yes, the compilers need to be taught about it)14:02
lkclexample, IIT Madras developed hardware pointer "tracking"14:02
lkclevery 64-bit memory pointer had a "mirror" tag (also 64-bit)14:02
lkclwhen memory was allocated, that "tag" was set to a hash.14:03
lkclwhen memory is freed, it's set to zero14:03
jnre. the Router funding, is this the right page? https://libre-soc.org/nlnet_2021_crypto_router/14:03
lkcltherefore (theory goes), if you access that memory after it's been freed, hardware checks the tag, and if it's zero, you throw an ILLEGAL MEMORY ACCESS exception14:03
lkcljn: yes14:03
lkclnow, there, it *would* be appropriate to teach the rust compiler about that featuer14:04
programmerjakedepending on the level of tracking, it could easily break fundamental memory model assumptions of C, Rust, LLVM and others14:04
lkclin fact, the IIT Madras team member told me that they were actually looking at rust to do exactly that14:04
lkclyes, they were *actually* designing the malloc-tracking *specifically* for rust, not just to help with c / c++14:05
lkclthat *IS* an appropriate circumstance to involve the rust language with actual hardware14:05
jnprogrammerjake: hmm, in that context, knowledge of Rust internals is useful while designing these HW features (pointer tracking etc.)14:05
programmerjakefor instance, Rust explicitly treats memory as untyped bytes, it doesn't have C/C++'s type-based-alias-analysis14:05
lkclhowever they got special funding to do so14:05
lkclthere are several other examples like that, where the compiler and/or language actually ties in directly with hardware features14:06
lkclRV has a JIT SIG where they are designing instructions to help accelerate all types of JIT (java bytecode, javascript, many others)14:07
programmerjakeso, in Rust, casting a `*const f32` to a `*const i32` and dereferencing is perfectly legal (though it requires an unsafe block), whereas in C that is UB14:07
lkclhowever when developing a general-purpose processor, you want to pick a *hardware* language appropriate for developing *hardware*.14:08
lkclwe're taking a huge risk by using nmigen, as it is.  fortunately it outputs verilog.14:08
lkcli spent some considerable time explaining to the team in India about the Dynamic Partitioned Signal14:09
lkcland illustrated to them how insane it would be to try that same thing in verilog or VHDL14:09
programmerjake:)14:09
lkclthey took it quite well, and got it14:10
programmerjakeit might be doable in VHDL due to it's nice Ada roots, but definitely not Verilog14:10
lkcli've put them onto the wishbone streaming task, because the initial work can be done in verilog14:10
lkclthat'll give them time to get up to speed14:10
programmerjakeit would definitely not be usual VHDL though14:11
lkclyehyeh14:11
lkclit's the operator-overloading in python, the fact that it's a full OO language, that makes it possible to do PartitionedSignal14:11
lkclthere's still things that will limit us, it's going to be quite awkward14:12
lkclbut it's possible14:12
lkclthings can still be expressed "m.d.comb += result.eq(x + y)"14:12
programmerjakehmm, operator-overloading isn't strictly necessary...as is evidenced by nmigen's a.eq(b)14:13
lkclthat's an annoying one, there's no operator-overload for "assignment"14:13
lkclpyrtl overloaded... what was it.... i think they overloaded the "<<=" operator to mean "assignment"14:14
lkcl(blech)14:14
jnit's a convenience feature, as "x + y" is shorter, more convenient than "x.add(y)" would be14:14
lkclmore readable, yes.14:14
programmerjakejn: yup!14:14
lkclesp. when you get nested expressions.14:14
lkclwe will seeeriously have to keep an eye on the gate count though14:15
lkcllook at the yosys graphs to make sure it's not totally insane14:15
programmerjakethough it's not always more convenient to write a + b instead of a.add(b)14:15
lkclironically, "x + y" ends up allocating a nmigen module "behind the scenes"14:15
lkclso the yosys graphs *really do* have boxes named "add", with 2 inputs and 1 output14:16
lkclrather than the usual oval with a "+" symbol in the middle14:16
programmerjakewell, yosys represents verilog's + operator using the `$add` cell (equivalent to a submodule) anyway14:17
lkclstill need the integration of PartitionedSignal into nmigen itself to be done14:17
* lkcl checking that, haven't looked at a yosys graph in a while14:17
lkclah yes it does14:18
lkclhowever the inputs are on the left and the output on the right14:18
lkclwhich makes for a pretty graph14:18
lkclforward-progress14:18
lkclwith PartitionedSignal, x + y calls x.__add__(y) which behind-the-scenes creates a PartitionedAdd module instance14:19
lkcland (sigh) modules have all inputs and outputs on the *left* side of the box.14:19
lkclthe graphs therefore look a bit of a mess14:19
* lkcl eep, it's 2:20pm and i haven't had lunch yet!14:20
programmerjakewell...I should get some more sleep...I woke up at 4am14:21
lkcloff to make noodles+veg. later all14:21
lkcl:)14:21
lkclyeahh that's a familiar one :)14:21
lkcli was up _until_ 4am lol14:21
lkclsleep well14:21
programmerjake:)14:21
ghostmansdAs for *f32 to *i32 cast, this is UB by standard, by in fact works and is widely accepted, and basically depends on compiler & architecture combo. I recall a lot of such stuff regarding unions. Even despite recent changes in Standard (was it C99? C11?), many still think type punning doesn't work. It works, but you must know for sure where and how you do it.21:19
ghostmansdIf you ask me, the whole aliasing brings more problems than it ever intended to solve.21:19
ghostmansdI can give another example: casting (void *) to function pointer. It is UB, again, because pointer to data might not be the same thing as pointer to code.21:21
ghostmansdNow guess what, have you met such platforms IRL for recent years?21:21
ghostmansdAnd have you used dlsym or GetProcAddr?21:21
ghostmansdSo it's UB per Standard, but certainly well-defined per superset standards (POSIX, WinAPI, whatever else crap they have these days).21:22
programmerjakecasting void* to a function pointer is implementation-defined iirc21:23
ghostmansdAnd actually UB has its benefits, especially performance-wise.21:23
ghostmansd6.3.2.3:8 A pointer to a function of one type may be converted to a pointer to a function of another type and back again; the result shall compare equal to the original pointer. If a converted pointer is used to call a function whose type is not compatible with the pointed-to type, the behavior is undefined.21:23
ghostmansdLiterally the first discussion I found. I'm too lazy and AFK to check it.21:23
programmerjakegcc (and clang and all other compilers that follow gcc) defines type-punning *through a union*. type punning via pointer casting is still UB21:24
ghostmansdMeanwhile it works.21:24
ghostmansdGranted that you know for sure where, when and how you do it.21:25
ghostmansdFor example, you disable aliasing completely.21:25
ghostmansdYes, C has a different mindset than Rust. I don't see pros or cons there, and for me this is not a reason to prefer one or another.21:25
ghostmansdHowever, rationale shown above looks fair.21:26
ghostmansdI mean, project-wise.21:26
ghostmansdFor local stuff, e.g. drivers in Linux, I have no objections, unlike some other kernel developers.21:27
programmerjakeyup, I'm not saying Rust's type punning makes it better than C there, just different. if not properly accounted for in designing CPU-level pointer checking extensions, you can break a lot of software21:28
ghostmansdBut the idea to rewrite the kernel itself would look like an anecdote to me, a purely essential for virtually one purpose: to prove that you can do it.21:28
ghostmansdYou can break a lot of software even writing it in Python.21:28
ghostmansdAfter all, "you can write Fortran in any language".21:28
programmerjakehmm, hope no one has to write fortran in BF :)21:29
ghostmansdI for sure think there might be enthusiasts even for this task :-D21:30
programmerjakewell, if you want to have fun with weird compilers, check out: https://github.com/xoreaxeaxeax/movfuscator21:31
ghostmansdI think starting an independent project in Rust might be OK. Some system service which is a res per se, independent from other things. Or maybe even shiny new libc. Or, well, even hobbyist kernel.21:31
ghostmansdI saw it, nice stuff. I mean, this is for pure fun, but hey, what's not.21:32
ghostmansdAt least they prove that mov is Turing-complete IIRC21:33
ghostmansdSo it has some educational value, if this is correct.21:33
programmerjakex86 mov is, mov on something like RISC-V or Power isn't21:36
ghostmansdI think anyone can learn Rust; the actual rationale to do it is unclear. This is debate like "Why Plan9 has never won over Unix, despite being awesome".21:36
ghostmansd(I don't really think it is, but hey, the authors likely thought it was)21:37
programmerjakewell, Rust has waay more adoption than Plan9 iirc21:37
ghostmansdThat's due to all this hype and large corporations starting to use it.21:38
ghostmansdSame with Go.21:38
ghostmansdBTW, the team is the same, and lot of concepts came into Go from Plan9.21:38
ghostmansdI mean, just look closer. Channels? Goroutines? These "interfaces"? Just compare Plan9 C library and compiler, and you see where it comes from.21:39
programmerjakewell, unlike Go, Rust does have some useful stuff that isn't in more mainstream languages21:39
programmerjake(not that Go isn't useful, but you can find all it's features in other mainstream languages)21:40
ghostmansdMemory safety? C'mon, some languages don't even have pointers, and some have immutable objects.21:40
ghostmansdThat's like saying "Haskell has useful stuff".21:40
programmerjakeyes, but immutable objects and GCs are slow at all the wrong times21:41
ghostmansdWell, even assembler has some cool stuff!21:41
ghostmansdOne might argue that you simply use it incorrectly. :-)21:41
programmerjakeyup, which is why any decent systems programming language supports inline assembly, including Rust21:41
ghostmansdThis sorta defeats "memory safety" mindset.21:42
programmerjakeassembly has stuff you can't do in any other way (ever tried writing a syscall routine in JS?)21:42
ghostmansdLol, this reminds me an old gag21:44
ghostmansdhttps://img-9gag-fun.9cache.com/photo/a576bNq_460swp.webp21:44
programmerjakeno, Rust allows you to use all the potentially dangerous (that can cause UB) stuff in delimited unsafe blocks, allowing you to not have to check all code for memory safety (the way you have to with C/C++), all that's needed is checking the unsafe blocks21:44
ghostmansdI think we walk in circles. :-)21:45
programmerjakeusing unsafe blocks, you can build sound abstractions such that people calling your code from purely safe code can't cause UB intentionally/unintentionally21:47
ghostmansdI don't feel the need to advocate for C, I already know why I use it, and there's no need to persuade anyone. Same for Python. Same for anything else. I have no idea why there's a huge count of fans which feel these needs for Rust. If it's so awesome, people will find their way.21:48
programmerjakemeaning the code that doesn't use unsafe doesn't need to be written carefully to avoid UB21:49
ghostmansdHowever, there are reasons to use languages other than Rust, and, regardless of personal beliefs, they do have right to be accepted, like them or not. I personally don't like makefiles for many thins, but do I go everywhere to advocate any meta-build-system? No, even though I personally find meson attractive.21:51
programmerjakeI guess it's because I (and similar-minded Rust advocates) see the abstraction and safety powers that Rust gives while still being a systems programming language and see it as a massive improvement over other common languages21:52
ghostmansdDon't fall into the fallacy to think others don't see.21:52
ghostmansdAgain, I feel we walk in circles.21:53
ghostmansdI have no intention to stop the bright future where everything is written in Rust.21:54
ghostmansdPerhaps I will use it for something one day.21:54
programmerjakeyup, I don't think others are blind, but I see it as such a massive step-up that I can't help advocating for Rust in hopes others will be convinced to try it out and see for themselves21:54
ghostmansdWho knows, maybe lkcl is now re-writing the whole project in Rust, he just doesn't want to admit it... :HD21:56
programmerjakehe seems to like Python...21:56
ghostmansdI don't like any language.21:57
ghostmansdWell, at some point I actually loved Latin, but that's another story.21:57
programmerjakeXD21:57
ghostmansdI don't like languages, I like concepts which I create using those. I might like some features or not, but, after all, languages are just tools with pros and contras.21:58
programmerjake(i have an unusual fascination with Japanese and Unicode...)21:58
programmerjakeI like Rust as a concept!21:59
jnlkcl's main point in promoting python for all of LS is the follow, as far as i understand it: 1. it is already the main implementation language of LS, due to nMigen;  2. it is fairly well known in general;  3. the benefit of an accessible programming language (due to 1. and 2.) outweigh the comparative advatages of alternative programming languages, esp. Rust21:59
programmerjakeyup, I understand lkcl's reasoning (afaict)21:59
jns/the follow/the following/21:59
ghostmansdUnicode is elegant in some places, e.g. how it handles Hangul compositions, but is extremely awful in others (e.g. these cross-references across various parts of the standard).22:00
jnso, all the benefits of rust are good and interesting per se, but don't attack his argument22:00
ghostmansdAlso, ICU is a total crap, which sadly can do anything Unicode contains.22:01
ghostmansdI mean, interfaces-wise, at least C and C++ versions.22:01
programmerjakeI quite liked reading about harfbuzz22:01
programmerjakeand all the mess that is font rendering22:02
ghostmansdI've never dealt with it, but can believe :-)22:03
programmerjakeI do kinda wish that there never was UCS-2 or UTF-16, UTF-8 is good enough for everything. UTF-32 and code-point indexing is just a misleading dead-end -- makes people conflate code points with characters (grapheme clusters in Unicode-speak)22:05
programmerjakewindows having separate Unicode apis is annoying22:05
ghostmansdWell, for Windows it's already a part of backward compatibility, there are pros and cons.22:07
ghostmansdBTW, UTF-8 is also from Plan9.22:07
ghostmansd9p, procfs also are22:08
ghostmansdSo the concepts found their way onto the outside world22:09
programmerjakewindows should have just followed unix and plan9 and switched their ascii apis to utf-822:09
ghostmansdMaybe that's Rust's fate, who knows22:09
ghostmansdprogrammerjake: http://utf8everywhere.org/22:11
ghostmansdI think you read it before, but anyway :-)22:12
programmerjakerust inspiring new features in other languages: that's already happening to some extent22:12
programmerjakeyup, read utf8 everywhere multiple times22:13
ghostmansdI think other languages inspire as well. I don't think coroutines were that hyped before go, even though they existed decades before.22:14
ghostmansd(I actually love the concept, so "hype" is quite in a positive sense)22:17
programmerjakeinterestingly, I implemented generators in a realtime JS interpreter I wrote for a previous job22:18
programmerjakeiirc the only major feature I left out was regex support (who needs that for emulating embedded devices)22:20

Generated by irclog2html.py 2.17.1 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!