| lkcl | https://superuser.com/questions/626810/how-to-get-apt-get-source-verification-working | 10:10 |
|---|---|---|
| lkcl | Las[m], justinrestivo[m]: ^ | 10:10 |
| lkcl | that tells us that debian does indeed use the (massive, real-world-identity-backed, verified) debian keyring / web-of-trust to verify the integrity of source code used for all and any packages. | 10:11 |
| lkcl | the reports there show how people are having problems due to bugs in gpg (and also reports that there is a huge amount of misinformation, some of which has quite serious security implications for the users running the random "blind-leading-the-blind" commands being "advised") | 10:13 |
| lkcl | also there is this: https://www.debian.org/devel/buildd/ | 10:14 |
| lkcl | which is a way that packages may be marked for auto-building (and their dependencies), originally designed for porters to keep up with other architectures | 10:15 |
| lkcl | aside from a few (really useful) areas, it's looking, unfortunately, like nixos is displaying NIH syndrome. | 10:16 |
| lkcl | the pip3 checksumming is brilliant, and extremely useful. | 10:16 |
| lkcl | where are the modifications to pip3 available? | 10:17 |
| lkcl | my feeling is that we should extract those and actually use them in the reproducible build dev-env-setup scripts. | 10:17 |
| lkcl | also, it would be fantastic to have the checksum modifications submitted upstream, such that the manifest file can be installed by OSes or end-users | 10:18 |
| Las[m] | It's not really possible without using Nix | 10:19 |
| lkcl | Las[m]: some of the programming that i've done over the past 35 years, i don't believe in "not possible" :) | 10:20 |
| lkcl | in 2010 i combined 4 different projects and used 12 different programming languages in 2 weeks to create 20,000 python bindings to webkit DOM, making it a FULL peer of javascript. | 10:21 |
| lkcl | "not possible" is just an invitation to prove that it is :) | 10:21 |
| Las[m] | There's no reason not to use Nix though | 10:22 |
| lkcl | is there a pointer to where the source code used for pip3 may be viewed? | 10:22 |
| Las[m] | You can just avoid using Nixpkgs | 10:22 |
| lkcl | does it have a GPG keyring backed web-of-trust for source packages, of over a thousand developers? | 10:23 |
| lkcl | for the *source* packages? | 10:23 |
| lkcl | not the binary packages, the *source* packages? | 10:23 |
| lkcl | ah, is this it? https://github.com/nix-community/pip2nix | 10:25 |
| Las[m] | It doesn't AFAIK, which is why you might choose to avoid using Nixpkgs | 10:25 |
| Las[m] | Yep that's one solution, and probably what we'd use for Libre SoC | 10:26 |
| Las[m] | FWIW, I've been thinking about it a bit, and even without cryptographically signing the commits, I can't think of a trivial way to tamper with the source code. | 10:33 |
| Las[m] | People would notice if the upstream hash suddenly changes incompatibly | 10:33 |
| Las[m] | The link to Nixpkgs: https://github.com/NixOS/nixpkgs | 10:35 |
| Las[m] | It's a huge monorepo as you can see | 10:35 |
| Las[m] | FWIW, I can't see how Debian signs their commits: https://salsa.debian.org/python-team/packages/python-promise/-/commits/debian/master | 10:37 |
| Las[m] | None of these commits are signed | 10:37 |
| lkcl | commits are not signed, the entire package source is signed. | 12:13 |
| lkcl | the reason is because packages may be from .tgz archives or other types of repositories | 12:14 |
| lkcl | in the case of git-buildpackage, a tar archive is automatically extracted, and, once available, a SHA checksum may be generated and the signature verified on that | 12:14 |
| lkcl | also included in the signature is the debian maintainer's debian/rules, debian/control and all other information required to build the package. | 12:15 |
| lkcl | it's much more sophisticated than people think [who don't know how it works, and consequently end up duplicating it] | 12:16 |
| lkcl | "people would notice" - how? | 12:17 |
| lkcl | if there's no automated "noticing" that's a black mark | 12:18 |
| Las[m] | You are right | 12:19 |
| lkcl | i.e. if there's no tool which can provably verify the upstream hash, then it can be tampered with in a way that causes damage | 12:19 |
| lkcl | yes people would "notice", but the damage would potentially already be done | 12:20 |
| Las[m] | But there is a high chance that you'll notice that the git commit you got is not what other people have | 12:20 |
| lkcl | if however each commit was GPG signed, by (any one of hundreds of maintainers) then not only could automated tools "notice", you'd also have a way to track who was responsible | 12:20 |
| lkcl | and issue a revocation of their GPG key | 12:21 |
| Las[m] | It's not like Debian where packages might be any arbitrary version | 12:22 |
| Las[m] | If we assume that the situation with signing commits will not change, what you'd do to ensure security is to pin your Nixpkgs dependency to a specific commit | 12:23 |
| Las[m] | Then you might, perhaps, once every 6 months, find another valid commit to change to | 12:23 |
| Las[m] | You could check that this is the commit that other people are using | 12:23 |
| Las[m] | It could then be that the source that everyone is using has been tampered with, but this is much harder | 12:24 |
| Las[m] | Since people are looking at the source code | 12:24 |
| lkcl | apt-get install package/version-number | 12:39 |
| lkcl | i presume also apt-get source package/version-number (although i never used it) | 12:39 |
| lkcl | and apt-pinning then ensures that version is not replaced | 12:39 |
| Las[m] | Yes but that doesn't check checksums | 12:40 |
| lkcl | yes it does. | 12:40 |
| Las[m] | AFAIK | 12:40 |
| lkcl | the checksums are what are GPG signed | 12:40 |
| Las[m] | Ah, really? | 12:40 |
| Las[m] | So a package/version-number is tied to a specific checksum? | 12:40 |
| lkcl | ... from the Release File, yes. | 12:41 |
| Las[m] | Ah, huh | 12:41 |
| lkcl | remember, debian distributes .tar.gz archives as well as git repositories | 12:41 |
| lkcl | e.g. most GNU packages are distributed as tarballs, not git repositories | 12:41 |
| Las[m] | Is the checksum checked when downloading or is it only the signature that is checked? | 12:42 |
| lkcl | https://wiki.debian.org/DebianRepository/Format#A.22Packages.22_Indices | 12:42 |
| lkcl | both. | 12:42 |
| Las[m] | Makes sense, that's nice | 12:42 |
| lkcl | http://ftp.uk.debian.org/debian/dists/Debian10.10/Release | 12:43 |
| lkcl | which is the GPG-signed http://ftp.uk.debian.org/debian/dists/Debian10.10/Release.gpg | 12:43 |
| Las[m] | And I assume this is also signed somewhere? | 12:43 |
| Las[m] | Makes sense | 12:43 |
| Las[m] | What I mean before though wasn't in this way | 12:48 |
| Las[m] | In your install scripts, you don't check the checksums of the libraries you install AFAICT | 12:48 |
| Las[m] | Or am I wrong? | 12:48 |
| Las[m] | I.e. the development environment isn't exactly reproducible | 12:49 |
| Las[m] | With Nix, you will always get exactly the same versions, unless you explicitly update the dependencies | 12:49 |
| Las[m] | Version being defined as checksum | 12:50 |
| Las[m] | The reason I bring this up is that this is also a security measure in itself, since even if there isn't a person to trust, you can check that the checksums match what everyone else is using trivially. | 12:52 |
| Las[m] | Although obviously signing is still beneficial, which is why I'm drafting an RFC to introduce that | 12:52 |
| lkcl | package dependencies are also packaged. | 13:07 |
| lkcl | therefore, inherently, if it's a dependency, it's also signed. | 13:07 |
| lkcl | and with the work for the past... (5?) years in debian on reproduceable builds, then, clearly, that's also possible | 13:08 |
| lkcl | (where do you think the work on reproducible builds came from, such that nixos could consider using it?) | 13:08 |
| lkcl | answer: it came from work done by redhat for RPM distros and by debian developers for apt/dpkg packages | 13:09 |
| Las[m] | It's not possible without something like Nix though | 13:09 |
| lkcl | that's incorrect. | 13:09 |
| Las[m] | Then how would you do it? | 13:09 |
| lkcl | debian (should, by now) have reproduceable build checking as part of dpkg. | 13:09 |
| lkcl | ah, found the status / progress page https://wiki.debian.org/ReproducibleBuilds | 13:10 |
| lkcl | "We have a new control file *.buildinfo that records the build environment" | 13:10 |
| Las[m] | How do you actually ensure the hashen of the dependencies you are using when developing/building Libre SoC though | 13:12 |
| lkcl | so out of appx 32,000 packages, appx 27,000 are currently reproducible | 13:12 |
| lkcl | we don't. we rely on debian/stable | 13:12 |
| Las[m] | And how do you ensure they correspond to the source code for building the packages | 13:12 |
| lkcl | and on manual git tags. | 13:12 |
| lkcl | we don't. | 13:12 |
| Las[m] | So effectively you only rely on trust | 13:13 |
| lkcl | because there's not enough people helping, therefore we have to trust the external dependencies | 13:13 |
| lkcl | yes. | 13:13 |
| Las[m] | You trust the Debian developers not to be malicious | 13:13 |
| lkcl | that, and if there's an external... | 13:13 |
| lkcl | of course. by way of their reputation | 13:13 |
| Las[m] | Whereas with Nix, you can just see that it hasn't changed and is correct | 13:13 |
| lkcl | as in: if they were ever caught acting maliciously their career is over | 13:13 |
| lkcl | that's incorrect | 13:13 |
| Las[m] | How so? | 13:14 |
| lkcl | the chain of trust goes back to the original source, as picked up by the debian developer | 13:14 |
| lkcl | and the debian developer takes responsibility for reviewing the source code (as mentioned yesterday, due to the copyright file, debian developers *have* to do a full source code audit) | 13:14 |
| lkcl | 85% of debian is reproduceable according to that graph, btw | 13:15 |
| lkcl | sorry, 85% of amd64 is reproduceable | 13:15 |
| lkcl | i have a meeting in 45m i have to prepare for | 13:15 |
| lkcl | Las[m], when writing that RFC this should be useful | 13:19 |
| lkcl | https://ftp.libre-soc.org/wot.pdf | 13:19 |
| lkcl | if the *EIGHTEEN* requirements are not met, the RFC should be considered, effectively, harmful. as in: if all requirements are not met, implementation of the RFC will result in a false sense of security. | 13:20 |
| Las[m] | Thanks | 13:20 |
| Las[m] | What FPGA do you recommend for testing Libre SoC? | 21:53 |
| programmerjake | I didn't actually get around to buying one yet, but I think we're planning on getting some ECP5-based boards made that have a decent amount of ram and other stuff, lkcl would know for sure | 22:25 |
| matthewcroughan | What page do I read to understand how to simulate the libreSOC? | 22:39 |
| lkcl | Las[m], _probably_ the easiest one right now is the ULX3S https://radiona.org/ulx3s/ | 23:21 |
| lkcl | although i've just got a Digilent Arty 100, in *theory* it should be dead easy to build / upload. | 23:22 |
| lkcl | we need the nextpnr toolchain for it though | 23:22 |
| Las[m] | What won't work without the nextpnr toolchain? | 23:23 |
| lkcl | matthewcroughan: if you just want to run simulations of the HDL, then run the devscripts and then pick one of the unit tests | 23:23 |
| lkcl | FPGA building | 23:23 |
| lkcl | for FPGA targets supported by nextpnr | 23:23 |
| Las[m] | So the board isn't usable without some work? | 23:23 |
| lkcl | which board? (there are hundreds) | 23:23 |
| Las[m] | ULX3S | 23:24 |
| lkcl | ULX3S "just works" | 23:24 |
| lkcl | https://libre-soc.org/HDL_workflow/ECP5_FPGA/ | 23:24 |
| lkcl | https://libre-soc.org/HDL_workflow/nextpnr/ | 23:24 |
| lkcl | https://git.libre-soc.org/?p=dev-env-setup.git;a=blob;f=nextpnr-ecp5-install;hb=HEAD | 23:25 |
| Las[m] | Thanks | 23:25 |
| lkcl | https://git.libre-soc.org/?p=libresoc-litex.git;a=blob;f=README.txt;hb=HEAD | 23:25 |
| lkcl | however, the larger Digilent Arty-A7 100, i have one but haven't had time yet to get *its* version of nextpnr built. | 23:26 |
| Las[m] | Ah | 23:26 |
| lkcl | btw get the largest FPGA you can. | 23:27 |
| Las[m] | Does it being larger make any difference for Libre SoC? | 23:27 |
| lkcl | Power ISA is *expensive* in terms of gates, due to it being a high performance ISA | 23:27 |
| Las[m] | Why? | 23:27 |
| lkcl | ohh yes | 23:27 |
| lkcl | because it's a minimum 20,000 LUT4s, that's without an MMU and FP. | 23:27 |
| Las[m] | So it uses as many gates as possible? | 23:27 |
| lkcl | Paul Mackerras added *partial* VSX and that required a staggering additional 30,000 LUT4s. | 23:28 |
| lkcl | which is one of the reasons we're not doing VSX / SIMD. | 23:28 |
| lkcl | sigh vivado is proprietary. frickin nuisance https://projectf.io/posts/hello-arty-1/ | 23:29 |
| programmerjake | later we will want to use a FPGA to run something with multi-issue, out-of-order, speculation, multi-core, L1 and L2 caches, etc. that needs a big FPGA to fit it all | 23:29 |
| Las[m] | The ECP5 only has "85k LUTs", is that not enough? | 23:30 |
| lkcl | ah ha! https://twitter.com/gatecatte/status/1163742441272487936 | 23:30 |
| lkcl | Las[m], it's enough... for now. | 23:30 |
| lkcl | i have a VERSA_ECP5, only 45k LUT4s, and 50% of it is used up by the *integer-only* first version | 23:31 |
| Las[m] | Thanks for the recommend then | 23:31 |
| lkcl | compare that to e.g. a "simple" RISC-V 32-bit core, that can be done in only around 3,000 LUT4s. | 23:31 |
| lkcl | 64-bit Power ISA is *massive* | 23:31 |
| lkcl | bleh. https://github.com/daveshah1/nextpnr/tree/rapidwright-xc7 | 23:33 |
| lkcl | java | 23:33 |
| * lkcl spews | 23:33 | |
| lkcl | there's another toolchain around somewhere. run by google so of the usual form "wget | bash". sigh | 23:33 |
| programmerjake | spilling your coffee? :P | 23:34 |
| Las[m] | Isn't that still better than doing it in Python? | 23:34 |
| lkcl | Las[m], "serious" low-level programming isn't done in java. | 23:36 |
| lkcl | ah ha! https://bugs.libre-soc.org/show_bug.cgi?id=654 | 23:36 |
| programmerjake | that's a point of disagreement...I think Rust would be better than Java or Python, but Java could be better than Python -- depends | 23:36 |
| lkcl | rust due to lack of adoption becomes a serious barrier to contributing. we've already made that mistake. | 23:37 |
| lkcl | https://carlosedp.medium.com/xilinx-open-source-fpga-toolchain-on-docker-containers-93202650a615 | 23:38 |
| lkcl | java requires insane 150mb+ runtime environments, even to run the tiniest program | 23:38 |
| Las[m] | Not anymore | 23:39 |
| Las[m] | It can be compiled AOT into small executables | 23:39 |
| lkcl | i heard gcc has a java frontend. the language itself is so anathemic i won't touch it. my only experience with java (a red5 plugin) it was as bad as i expected it to be, haven't used it since | 23:40 |
| lkcl | https://carlosedp.medium.com/xilinx-open-source-fpga-toolchain-on-docker-containers-93202650a615 | 23:40 |
| lkcl | ngggggh dockkkhaaa | 23:41 |
| lkcl | that's where i remember the "wget googleurl | tar -x | bash" comes from | 23:42 |
| lkcl | for symbiflow | 23:42 |
| lkcl | ngggh stress! :) | 23:44 |
| programmerjake | do note that Rust is more popular than you might think, according to the 2021 stack overflow survey Rust is known by a little more than 7% or respondents -- about the same as Go and more than Perl and Assembly and Matlab | 23:45 |
| programmerjake | Rust is also growing quickly | 23:45 |
| lkcl | programmerjake: none of these things make it appropriate. "quickly growing" means "unstable". | 23:48 |
| lkcl | we've learned this lesson already | 23:48 |
| lkcl | https://aur.archlinux.org/packages/symbiflow-arch-defs-nightly-bin-device-xc7a100t/ | 23:48 |
| programmerjake | umm...quickly growing doesn't mean unstable, Rust hasn't had any noticable breaking changes in 5+ years | 23:50 |
| programmerjake | in fact, they're specifically intending to not ever have breaking changes | 23:51 |
| lkcl | programmerjake: i'll be absolutely honest with you, here: the biggest alarm bell regarding rust is how much you promote it, and how you won't do any programming in anything other than rust. | 23:52 |
| lkcl | or, as you pointed out on libre-soc-misc, you'll do "non-rust" programming "if absolutely forced to". | 23:53 |
| programmerjake | well, that's because I honestly think it has critically important benefits that all the other mainstream languages missed | 23:53 |
| lkcl | this has been a huge concern for me for over 18 months, now. | 23:53 |
| lkcl | i know that: it does not automatically mean that "all other languages are total s*** and not worth bothering with" | 23:54 |
| lkcl | more than that, the language *itself* is not "the ecosystem". | 23:54 |
| programmerjake | I never said they weren't woth bothering with or that they were trash | 23:55 |
| lkcl | just because the *syntax* is "great for a given specific purpose" does not automatically and inherently mean that all other languages are... | 23:55 |
| lkcl | you did indeed. 1 sec... | 23:55 |
| programmerjake | I just said I think Rust is better in almost all cases | 23:56 |
| programmerjake | being better than a thing doesn't mean that thing is bad | 23:56 |
| lkcl | it has in practice meant that you've not contributed to the nmigen codebase in over a year, and before that it was somewhere around 6-8 months to the previous contribution | 23:58 |
| lkcl | which has put extreme pressure on me as the sole exclusive full-time HDL and Simulator developer | 23:59 |
| programmerjake | that's not caused by my liking Rust -- that's caused by my other problems...sorry | 23:59 |
| lkcl | i know. i get it. i've been there (intriguingly, around the same age as you are, now) | 23:59 |
Generated by irclog2html.py 2.17.1 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!