unfortunately it's a bit annoying to get the necessary pins available to manipulate. i have a digilent cora and arty board and neither of them actually pin out everything, which is really annoying. fortunately if you've been keeping up with xilinx shit, there was in the past a huge flood of these buttcoin1 mining control boards on aliexpress and similar sites which contain a zynq 7010 (at the time, pretty significantly below cost -- was a cheap way to get normally expensive fpgas)
specifically the EBAZ4205, which has schematics available here: https://github.com/xjtuecho/EBAZ4205
luckily, this board is really conveniently laid out, probably completely stolen from some sort of secret xilinx reference design that you only get if you are a Real Customer, so on all the bootmode pins there are actually pairs of pads for pull up or pull down, and all you need to do is swap the resistors to the adjacent pad to switch the mode. and the uart pins needed (MIO 48/49) also have accessible pads near the chip
XLNX-ZYNQ
use the code here (you may need to make some modifications) to execute the bootrom self-dump, and then use jtag (jean tag) to read out the code from the start of ram2
that should be it. enjoy yr bootrom
if you're interested in learning more about cryptocurrency, see this post
it was discovered that you don't actually need a xilinx jtag cable or specialized hardware for this (hardware vendors love to scam you out of literally hundreds of USD for a fucking cable. fucking sucks). you can use a j-link (or probably any jtag device, tho i didn't test anything else) and just make sure to check the pinout and connect the lines the right way from the board to the debugger. iirc xilinx swapped some shit for no reason (that's how they get away with selling their special cable i guess). then on the j-link console do savebin as in savebin bootdump-zynq7010.bin 0x0 0x20000
part 2 of the season released on friday, that's 8 more episodes that complete the season (no news on whether there will be more at the moment...)
quick recap:
in episode 9 we have leverage pulling an unusual job... an elderly small town librarian has cancer, and the team is tasked with setting up a fake spy-thriller adventure for the guy just like in the books he likes to read (complete with hacker parker,,, with orange soda!!! adorable imo). problem is... he turns out to be an actual spy. oops. also RIZ, the private security force we saw earlier in the season is sending their men to kill the librarian spy and grab an important piece of intel he's still holding onto (in eliot's words, of course, "mall cops")
episode 10 is leverage vs the #girlboss. an evil social media influencer selling fake medicine to desperate people. also in the process they meet this comically evil caricature of a bitcoin CEO dudebro who is funding the operation (and breanna wipes all his bitcoin at the end. based imo)
then there's episode 11
episode 11 was a lot
the opening is a standard bad guy sort of deal. Elder Abuse LLC is conspiring with a corrupt judge (redundant phrase, i know) to get guardianship over tons of senior people and then basically take all their stuff and their house. we're introduced to stella, who has alzheimer's and is struggling to save all her stuff while the guys, having full legal authority to break into and loot her house are doing so. this sucks. i'm already mad. it gets worse. stella had someone she had a relationship with in the past sign a power of attorney, which could get rid of the Extremely Punchable Faces LLC guys for good but alzheimer's made her forget who it was or where the document is stored ;____; gets even worse though. stella turns out to be the world's greatest grifter (back in the day at least): The Jackal. even beating sophie in one con. literally famous among grifters everywhere. also, stella is gay (new guy: didn't see that coming. breanna: yeah don't take it personally most straight guys never do). on a con against a london gangster, she accidentally fell in love with his wife and they ran away together to New Orleans and raised a child, in hiding, since of course gays have no rights, signing each other's power of attorney since they couldn't actually marry. meanwhile, of course by law the "husband" still has all the legal power so he tracks them down and shows up one day. very mad at the "degenerates". he tries to kill stella but stella's wife shoots him first. stella takes responsibility for the shooting and never sees her wife or kid again. anyway i'm nearly crying by this point. but as a consolation leverage does manage to reunite stella with her wife and child and parker pulls a favor to get the judge and guardian bonked. all is well, presumably, but stella still has alzheimer's. homophobic imo
episode 11 was supremely heavy, and it was kind of surprising. at the same time though breanna did say "people like us" when referring to herself and stella, which i think means she's actually gay. i just don't know why she doesn't just like say that out loud. stella is allowed to be openly gay and the client from the librarian spy episode basically opens with saying he's gay but for some reason breanna has to be a crypto-gay. it doesn't really make strategic sense because i doubt anyone that would be legitimately mad abt that is watching the show i would think. it's weird. i originally thought breanna being gay was something they just dropped out of the script but i guess not
there's another aspect which is getting to me about redemption compared to the original leverage series which is that some of the leverage enemies here are just like, more fake somehow. the villains are almost cartoonishly evil. a caricature of some of the worst parts of society but constructed in a way that they have no actual depth besides being caricatures. it's over-the-top obvious what they're supposed to represent and thus the show's stance on those kinds of people, and honestly i think that detracts from redemption a bit. take the bitcoin guy for example. compare him to Dubenich from the original series, or Damien Moreau, or Jack Hurley, or especially James Sterling. all these guys had at least some depth to their character. they're marks but they're not just flat or one-sided. they represent various evil aspects of society that leverage is fighting against, but they're also actual people on top of being those representations. meanwhile bitcoin man is literally just a caricature of the stereotypical rich techbro and nothing else. like the whole sequence in his office felt like a saturday morning superhero cartoon. i like how the show obviously seems to agree that bitcoin bros are not good, but the fakeness of the whole sequence of interactions was just super weird. idk how to describe it.
sophie: parker! where did you get that
parker: hmph. i bought it
yea that's all i have for now. see u in the next leverage post :3
]]>one of the fucky things that can happen when you use two computers is you can end up in a situation where you have a microphone on one computer but need audio out on the other one, particularly if you're in a call and playing a game. one solution to this is just join the call twice, on both the computers, one for audio out and one for audio in. this kind of sucks though. turns out there's a better way!
scream is a network audio driver meant for vfio windows. especially in combination with looking glass, though it can work on its own too. vfio is also not a strict requirement, but i'm not sure that people usually want to be doing network audio on bare metal windows. basically it takes your desktop audio and spams it out as PCM on the network where it can be picked up by any other device using a magic multicast address. using multicast means it's effectively plug and play, without any configuration needed. i installed the driver and the receiver and immediately started getting audio, which was nice
to install the receiver on linux, you can use the official github releases, or get it from the AUR. then run: scream -o pulse
(assuming you have pulseaudio or pipewire)
to make this better you can also set up a user unit in systemd: ~/.config/systemd/user/scream.service
[Unit]
Description=Scream network audio receiver
[Service]
ExecStart=scream -o pulse
[Install]
WantedBy=default.target
and make it start and stop automatically depending on whether the computer (for me, a laptop) is on the home network (with the desktop on it): /etc/NetworkManager/dispatcher.d/99-scream-service
#!/bin/bash
if [[ "$1" =~ ^(enp|eno|eth) ]] && [[ "$2" == "up" ]]; then
systemctl -M <you>@.host --user start scream
else
systemctl -M <you>@.host --user stop scream
fi
adjust the script accordingly. here i just check if it's an ethernet device, because for my use cases ethernet means home network. you may want more advanced checks though
this also took me a while to figure out, but you can see here the canonical way to control a user instance of systemd from the root user -- because networkmanager invokes your script as root by default (actually the way i found out is systemd tells you what the syntax is when you try to use -H instead of -M). you use -M <your username>@.host
, and that connects to the systemd user bus for that user, assuming you have permissions
in the process i also discovered another fun thing which is that the systemd help outputs actually contain clickable links, which you can click in GNOME terminal (try it!). running systemctl --help
and if you ctrl-click the man page link at the bottom, it actually opens GNOME Help with that man page. pretty neat imo
escape codes to achieve this are basically \x1b]8;;<url>\x07
. for example
printf "\x1b]8;;https://awoo.systems\x07meow meow meow\x1b]8;;\x07\n"
in case you weren't familiar, GNOME terminal and most modern terminals support bold, italic, and underlined text as well as 24 bit color, color emojis, and different cursor shapes (block, caret, and underline -- which i set up to indicate vim editing modes). so there's a surprising amount of rich text capability which is why I like GNOME terminal (and Konsole though I use that less -- and I'm not sure if it supports links, though it has the other stuff) over many other terminals
anyway that's all i have. try scream if you need windows audio shipped to a linux device. the latency is plenty good enough for gaming (on gigabit ethernet)
]]>eta-git
) is a terminal command that can be used to monitor the progress of literally anything
how does it work
eta takes 2 arguments, the first is the total number of things to be processed by whatever you're tracking. the second is a command to run to get the current count of things that are done. both the total number argument and the command output will take the first number in the provided text, and ignore any trailing garbage -- this allows you to use things like wc
even though it normally outputs a count with a filename
that's basically it. the readme for the project has a bunch of examples of usage you can reference. but in general it's nice to have a tool that you can meme for any sort of ad-hoc progress for nearly arbitrary stuff you might be doing. here are some things i have personally used eta for
eta "$(fd -e c | wc -l)" "fd -e o | wc -l"
for url in $(cat ../urls.txt); do wget --content-disposition "$url"; done
eta "$(wc -l ../urls.txt)" "ls | wc -l"
cp -r
... yeah you get the basic idea, this would be like eta "$(du -bs $src)" "du -bs $dst"
basically, eta
is pretty underrated imo. try it out next time you need real time progress for something
so anyway ordering the screen was straightforward. the main issue is it takes a while to ship (from hong kong), and while being able to order parts for your phone can help reduce e-waste there is also the aspect of there presumably being a whole plane to fly this out across the world but i guess it's an improvement at least. i assume lots of people have pinephones in the US so it may make sense to just ship a bunch of parts out and store them in the country so it's easier to deliver when people need them here. anyway after a week it got here
the repair part... turns out really all you need is a small phillips screwdriver. it helps to have a spudger but you can do it all by hand too. though to be honest i would really prefer if they used torx screws instead of phillips because phillips is really easy to strip. i think the convenience of not having to get a torx set which i guess most people don't have is not quite offset by the risk of stripping phillips. torx is literally just better. but all pine64 devices seem to use phillips anyway which kind of sucks
the phone has a back cover which can be removed by hand, then an inner frame which is screwed in, then the mainboards, components like cameras and speaker, and some ribbon cables, and this all sits in the front chassis assembly which includes the screen. you get that whole front part as a single unit with the screen assembled into it already. so it's just a matter of removing every component in order and then assembling them into the new front chassis/screen piece. i even almost got this right the first time! i found out i forgot to move the top speaker at first. there are a lot of small pieces so you have to make sure you have them all
also, i discovered yet another design flaw during this process and i'm not entirely sure how to reliably fix it. the bottom speaker output had not been working on my phone for a while. i assumed it was potentially a software issue but it turns out when i looked at it, the bottom speaker connects with a ribbon cable to a side board which contains the usb-c connector and some other things. the problem is, unlike every other ribbon cable in the phone which was attached with mechanical connectors, this one was soldered directly onto the board. and the other problem with this board was that the only mechanical attachment to the chassis was with these plastic pins on the chassis that go through holes on the board -- and the pins are too small, so the board can wiggle in place. which particularly becomes a problem because it'll wiggle every time you plug in the USB. so the end result was the solder joints for the speaker just broke. and..... yeah i need to find some way to mechanically secure this board despite it being probably the most regularly stressed part of the whole phone, and then solder the speaker cable back. or something. idk what i'm going to do yet
this is kind of an example of yet another design issue that can only be discovered by actually using the phone for a long time. or idk, being smart and realizing the usb board is not a good place to attach things. or making those pins wider. literally anything. on one hand it's nice that this is literally the only phone which you can actually repair easily but it's also not a good phone and it's fundamentally unsalvageable without a major hardware revision imo. oh well. maybe the next generation of linux phone, whoever ends up making them, will be better
]]>i ended up watching the released episodes approximately one episode per week, except for the end where the friend i was watching it with and i decided to just barrel through the last 3 at once or something. now, part 2 of this new season is coming out this month so despite it having been a few weeks at this point since part 1 i wanted to provide a quick reflection
first of all, it was good. i want to preface with this because while these days i don't really end up watching a lot of movies or TV in my free time, i did specifically set up time to watch redemption with the expectation that it would be entertaining and it was in fact a good watch
anyway be warned: spoilers below!!!
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
one obvious thing is the style is slightly different than the original. there's new music, the cinematography is slightly different, and of course since the returning characters have been growing for years in the meantime, the character dynamics are evolved (but there's now breanna and the lawyer guy to mix things up). the thing about reboots like this is that unfortunately they would never be able to be exactly like the original was. but regardless, i was really happy with the way the show turned out
so let's talk about the missing characters. nate is gone (not going into this right now but let's basically say hatsune miku played nate in the original1), and the canon is that he passed away. the first episode kind of gets right to this with a scene where sophie is in the graveyard at nate's grave, but we never really learn what happened. in the meantime, we get to learn some interesting things about nate posthumously, particularly in the last episode of this segment, where a former IYS Insurance accountant who was good friends with nate talks about how nice nate was to the accountants, who were basically the underground heroes of the operation even though the insurance agents tended to get all of the credit for catching fraud. which is obviously a big contrast to how nate treated the team in the original, and it's played to some comedic effect. more importantly nate apparently trusted this accountant guy enough to basically tell him everything about the leverage team, which also sucks because the main conflict of the episode is the guy basically trying to publish a book about leverage which would unfortunately seriously threaten the team if it got out.
meanwhile sophie effectively becomes the new nate. she starts to basically lead all the operations, and she does end up being quite good at it. even with 2 new members of the team, who are kind of both off the rails in their own ways. at the beginning, it takes a lot of effort for the crew to convince sophie that they should start stealing things again (parker: we should steeeaaaalll something) but sophie does get really into it eventually
hardison is also missing, and with hardison goes one of the funniest character dynamics in the original series between him and eliot, which is sad. on the other hand breanna is adorable. i'm not sure i have that much to say about hardison's absence other than the canon reason is kind of contrived but whatever. breanna is very different than hardison, she's not just someone cast to fill in that missing role and act exactly the same, instead she brings a lot of her own personality and comedic dynamics to the team on her own and is a major reason this reboot is so good in my opinion. one of the main things breanna brings around episode 3 is this monologue that kind of explains her motivations that i think generally resonates with a lot of zoomer types (including an dragon). she's just like, i'm here because the world sucks. i lived through two recessions and also literally the nazis are back and i want to do something about this. and in fact she does, like these episodes have the team take out some of the absolutely slimiest business side handlers the world has to offer tbh, between a construction CEO who cut corners to save a quick buck and killed a bunch of workers in a building collapse, a smug ass pharma exec who profits off children dying (and he's just so fucking insufferable about everything, i feel like his whole character was made to mirror martin shrekli2 or something so it's really satisfying when he gets owned and of course the children get the lifesaving treatments they need), and there's also this techbro who appears to be a zucc mirror, profiting off deploying mass surveillance in a town. so breanna does get to make a difference, and gets into her role on the team very quickly. plus breanna and parker end up having some cute interactions
and then there's the returning characters. parker is still adorable and of course having an obviously nd character be treated nicely in a tv series is really good to see
sophie: ok so your social strategy is going to be-
parker: don't worry i have cards. see? if the other person says what's on the front of the card then I say what's on the back
sophie: parker you can't just reduce social interaction to a flowchart
sophie: .....
sophie: ... ok these are good though ...
mood tbh
eliot is somehow more of a model for positive masculinity than before. and tbh eliot is really good and i love that the main macho punchy action guy gets to be eliot because besides being the macho punchy action guy he is also capable of a lot of emotional maturity and he knows how to cook (guys that can cook? impossible). during one episode where he's sent to infiltrate the security guard operation of a riverboat, he ends up bonding with the head guard and making sure personally that he ends up safe at the end. plus the guy makes eliot some carrot cake
one of these days i am going to literally make the eliot sandwich. i want to try it tbh,,,,
on the new lawyer guy, i think he's also good. unfortunately the problem is he was played mostly towards the reboot's "redemption" theme, and as such didn't really end up being super memorable on his own. he's mostly a blank slate, trying to learn about how the team operates and this being played to some comedic effect sometimes (especially when he doesn't understand the classic memes from the original series). i don't even remember his name though, and i'm too lazy to look it up right now so that's why he's "the lawyer guy" which is kind of unfortunate. i guess it's kind of interesting how the main theme of the reboot revolves around the guy that i honestly have the least commentary on. though ultimately it is of course a really good thing that he's trying his best to atone for the sins of his past life. i think he'll make a good thief, eventually
so..... that's basically it. for now. the second half releases next friday and i'll be trying to watch one episode per week starting then. look forward to more livetooting, at least. i think it's going to be really good
she's so talented,,,,
he doesn't deserve to be spelled correctly
ok so basically there has also been august and september and tbh i don't think a whole lot of anything actually useful has happened
in august i went to DEF CON (registered trademark symbol Hacking Conference) which was massively scaled down because of covid but at least they attempted to do an in person thing which was still good tbh. more importantly i visited meow wolf's Omega Mart which was actually really good (honestly i'm willing to say the defcon trip was actually for omega mart, and defcon just happened to be going on at the same time ๐). omega mart is this sort of immersive art installation / mini ARG which you can see from the website is kinda .. well... there's something going on. i'm not going to spoil anything. it was a very very fun way to spend a couple hours of a saturday night (and then i stayed up the rest of that night trying to be the literally billionth person to hack the badge). going to stuff like defcon is always inspiring in terms of thinking about what kinds of tools and techniques i should be focusing on in order to improve the state of hacking. like, i want to do it faster, smarter, more efficiently, supporting workflows that are more ad-hoc and it always seems like that's potentially within reach. though since then i haven't really done anything or acted on any of the inspiration
recently i started a new modded factorio game. i wanna try to slowly get through it, though with the angelbobs modset it is going to take a long time to get anywhere. there was also the fun time of dealing with setting up the headless factorio server on my server, which took a lot of fiddling but it does seem to work now
there's some other server maid tasks that need to get done. i want to move my ghidra server to the R710 and set up some sort of SSO-enabled authentication which would make it more convenient to use. jellyfin also still needs SSO, and i guess it would be nice to have etherpad moved as well but tbh that is kind of working fine on the apu2 where it is currently located. finally there's some data i need to sort through from the latest backup of important phone files that happened before my previous phone died, which i haven't done yet because the data is in a super annoying format. basically what happened with the phone is it kind of stopped working all of a sudden, like, one morning it would just not power on. i have inspected everything inside and run a lot of troubleshooting but nothing seems to be working. it won't even boot the qualcomm recovery mode, it just seems totally dead. i suspect this is a result of water damage over time since a few years ago i broke and subsequently replaced the glass over the phone's cameras, which may have been a botched job and allowed water to get it. though ultimately i have no idea what's going on with it. meanwhile i have been using the pinephone full time which i continue to do while i "look for a new android phone" despite the pinephone kind of sucking. basically i don't recommend it but it's what i'm doing right now. recently i broke the pinephone screen (i have this feeling that maybe the pinephone might just be more fragile than other phones. idk if that's well founded though. it's kind of hard to stop screens from breaking at the end of the day) and discovered the process of buying replacement parts for the thing and doing the replacement, which is surprisingly easy and most importantly pine64 will not sue you for repairing your own device. so that's one thing the pinephone has going for it, and it's really nice actually in a world of totally intentionally unrepairable phones everywhere which is definitely making things harder for people and generating lots of unnecessary ewaste. i may talk more in depth about the pinephone repair experience later. no promises though
i'm trying to get through software foundations (particularly logical foundations, and understanding coq) in part because my undergrad curriculum had this awful computers theorems class where i didn't really learn anything useful, and i feel like i actually need to know this stuff. coq is based on ocaml and made by the same people who made ocaml (the french national lab INRIA) and it's a theorem proving system that allows you to formalize basically any proof, checking that it's actually valid, while also making it easy to develop complex proofs using defined proof automation techniques (called tactics). coq was used to build CompCert, a formally verified C compiler, as well as develop one of the modern proofs of the 4 color theorem. and despite being a certified thembo i'm getting to be able to solve a lot of the challenging exercises in this book pretty quickly so that is good imo. it's also kind of weird because the whole book is written in coq, like the chapters are based on coq source code which has been rendered to web pages with coqdoc, such that the comments show up as markup and the code show up as code blocks. so with this you work directly in that file basically, you download their source package and write up the solutions in the source as you read the actual content of the book in the comments. i just thought that was kind of interesting
there was actually a CTF somewhere in the past few weeks .... sometime .... and i made some writeups i think this one is probably the most interesting (also it's pwn and like, pwn is My category) it explains as an aside how to be able to write C that gets injected into a target process even though it wasn't strictly necessary here because you could have just shellcoded everything tbh. but it's still a useful thing to do when the stuff you need is too complicated to write in assembly and there's no pwntools wrapper for it either
doing more ctfs would probably be good but... motivation...
with october comes spooky season, which is always a welcome part of any year, as well as a special spooky season treat which is the second part of the new leverage season. i'm pretty ameowbongo about that and i'll definitely be posting abt it when it releases (maybe [no promises])
]]>no?
well here's how anyway
you will need
list all the modems to get your modem ID
sudo mmcli -L
set up the message (there are 2 steps here for some reason this could just be one step but whatever)
sudo mmcli -m <modem id> --messaging-create-sms="number=+1234567890,text='hello world'"
you should get a message like
Successfully created new SMS: /org/freedesktop/ModemManager1/SMS/<X>
now, take the SMS <X>
number and send it
sudo mmcli -m <modem id> --sms <X> --send
that's basically it. yea
]]>so where to go if you are craving a game that delivers being actually both cyber and punk? imo, half life alyx counts as cyberpunk, and it is also my literal best game of all time but that's, y'know, half life
on the other hand, there is this tabletop RPG (disclaimer: i have never played actual tabletop RPGs lol) called shadowrun, and while many people complain about the 5th and sometimes 4th editions of this game there is also a series of digital singleplayer versions of the game, the most recent of which was produced by harebrained schemes called shadowrun: dragonfall2
to make a long story short i absolutely adore dragonfall. if you're unfamiliar with shadowrun, the basic premise is that in the future the world has developed magic while also being a typical cyberpunk dystopian hellscape with dark cities and megacorporations ruling the planet, and shadowrunners doing crimes and staying hidden in the shadows, but also magic and races of people besides humans (elves, orks, trolls, and dwarfs, collectively called metahumanity), and dragons. like typical RPGs, your character has attributes that determine skills in various areas like, strength, quickness, skills with various guns, magic, and of course decking (breaking into computer systems in the matrix) and rigging (controlling drones), among others
dragonfall puts the turn-based RPG system into a video game pretty effectively. you control a team of yourself (a fully customizable character) and your team, consisting of semi-NPCs which you can acquire limited upgrades for and control in combat, but who also have their own dialogue that they will say when you talk to them outside of missions. dragonfall is very much like a visual novel -- there are rich conversations to be had with pretty much every character, even side characters in your home base, and your decisions affect the progression of the story (there is, i don't think this is really a spoiler, a "bad end" which you can get yourself into only if you are egregiously morally bankrupt, which i should hope you aren't lol). but a lot of the decisions you have to make play on subtler moral decisions, and you're left at the end questioning whether you really made the right choices (or whether you really had a choice to begin with). the main part of the game is the missions, which have you break into various corporations and strongholds to advance the plot, with some fairly complex strategy needed in order to defeat the enemies in turn-based combat as you plan actions for your whole team at once. in between missions, you can interact with your safehouse, the kreuzbasar, which is an anarchist autonomous zone (with a pretty sick flag imo, it's a bear with a rifle) somewhere in the middle of berlin where you're able to accept new jobs and side quests and interact with all the NPCs, as well as purchase new equipment and upgrades
the really compelling thing about this game besides being a pretty chill way to experience what is kind of a fighting/action game at its core in a turn-based isometric 2.5D format (rather than, say, 3D FPS, which while i love to play FPS games too can definitely be more stressful and require more energy to play) is the really excellent music and artwork that goes into each environment. there are several mainline missions and several more side quests you can go on each with very detailed cyberpunk environments overlaid with a banger soundtrack that really makes this game a whole cyberpunk experience, the kind of plot-driven RPG where you can really immerse yourself in its universe like you're the protagonist in a film or a novel, with the sound and visuals to make it truly immersive (although, it is not VR!). that is really why i highly recommend this game, especially if recent developments in the cyberpunk media genre have left you a lot to be desired
it's called an AAA game because when you learn how it was made it makes you go AAA
there is more by harebrained schemes that came before, particularly shadowrun: hong kong so i will kind of be playing these in reverse order. hong kong is reportedly "not as good" as dragonfall but we'll see after i play it~
Leverage is a crime drama. but not the kind you're probably thinking of. rather than following around boring police detectives solving murders on the literal thousandth literal cop indoctrination show to ever constantly air on every American television 24/72, leverage is about the bad guys. committing crimes.
every episode follows the same basic format: there is a large corporation or rich billionaire committing totally legal injustices on ordinary people, who come to the leverage team for help. the team runs a con, tricking the bad guys or pulling a heist, and often very creatively bringing justice where it is deserved. for example, one episode has an agricultural megacorp about to intentionally release a blight to kill off their competition (and create mass famine in the process), another has a mining company cutting safety measures in the mines to save a quick buck, or a pharma CEO trying to sell a medication by covering up the evidence of its lethal side effects
so basically, it's a show about extralegal mutual aid, in the form of heists and grifting and hacking and a bit of beating up the bad guys, and what i really like about it is that it doesn't take itself too seriously. it's a comedy-action show, and the character dynamics are honestly the best thing about it. the characters are, originally,
the main thing that baffles me about leverage is how this was allowed to air in the first place. i mean with all the usual cop stuff going on it's honestly really strange to see this random completely different show, which is about committing crimes to do good, and sticking it to the rich billionaires and CEOs and corporate overlords who never suffer any consequences for their crimes. turbo anticapitalism, i mean the central theme of the show is basically, as nate says, "sometimes bad guys make the best good guys," which really means that in a world where lots of crimes against ordinary people is legal and the bad guys get away with it, sometimes you need "criminals" to set things right. there is a scene where nate is talking with one of the villains of the show, his former coworker at IYS, james sterling, where they talk about how rich people crashed the economy in 2008 (they really did!), and when the government and interpol (sterling becomes an interpol agent) went to investigate how that happened they found literally massive amounts of corruption and shady business, the white collar crime that screws over ordinary people that nate's team is fighting against. and in this story, what interpol did instead of go prosecute all this shit that they uncovered is just put it in a book and hide it away on their secure servers, and never tell anyone what happened. sterling says that this is the way the world works. the system depends on these guys who deserve to be prosecuted, and if interpol actually went after all of them the system would fall apart. sterling and interpol is painted as the major antagonist as they want to basically protect the billionaire megacorp status quo, in a very suspiciously relatable scenario about the actual 2008 recession (and nate says, well, see, if you're not going to prosecute them i am going to steal your book and do it myself lol, with my team of "criminals" who commit "actual" crimes) and someone at TNT was like, yeah, this might be slightly the kind of thing a modern McCarthy might be scrambling to flag, but,,, we're going to air it anyway... and i think that's a little surprising. but what's more ironic is redemption is now released by imdb, which as you might know is Amazon Prime Video 2 Electric Boogaloo. and you might be thinking like, wouldn't jeff bezos be the kind of guy the leverage crew would be after? (and yeah, i'm also thinking the same thing so what is up with that lol. like. bezos paid for this. wtf)
so, go watch the original series and everything but let's talk about redemption. on friday i watched the first 2 episodes (and also, discovered that my jellyfin syncplay has some bandwidth problems, which are most likely caused by my server being ultimately connected to the internet via 2 hops over the same MoCA interconnect, and MoCA supports like 100 mbps max or something like that, which was kind of a problem. it might be worth ditching jellyfin syncplay and going back to the old setup where i streamed rtmp with ffmpeg at my VPS gateway that has the bandwidth to actually support streaming that. or yknow, my server deserves 10G ๐ฆ)
redemption brings back the leverage, almost 9 years after the original series ended, and for full disclosure after i caught wind of the casting call for one of the new characters over a year ago i have been pretty patiently waiting literally all year for this release so i am extremely hyped that it is finally here, but watching it slowly instead of binging all at once (so at the time of writing i've seen the first 2 episodes, which are cohesively kind of 1 large job). and i am going to tell you right off the bat that redemption absolutely does not disappoint. i don't have a whole lot to say about it yet other than that i am really glad to have this show back, though the crew has aged, nate is not returning, and hardison is canonically off to pursue leverage international, the worldwide operation that is keeping him super busy (also cause y'know his actor isn't here full time, but it's about leverage international), but we have the same basic format of comedy heist, the same memes (which cause some pretty funny confusion with the new characters) and of course 2 brand new characters who are here to provide (hopefully) new dynamics and also the eponymous theme of this reboot
so tbh without revealing anything about what actually happens in these episodes (i'll talk about it later, and it's also with some spoilers as marked in this threadโ) i am super excited. like, leverage is back, the reboot doesn't suck, and it's going to be great. the new characters are hilarious and it's going to be good following their character development during this series. i'm looking forward to it
โ* if you would like to watch with me (and maybe deal with my server falling over lol) it's probably going to be friday evenings EDT
definitely
i wonder why they always play those shows tbh
probably her real name. nobody really knows
just parker
one of the issues with doing time-domain side channel analysis is that all your traces need to be perfectly aligned. this can prove challenging, especially if the target has side channel countermeasures that include random delays in the sensitive code to throw off alignment. what can you do about that? well you could do your analysis in the frequency domain instead of the time domain
typically side channel traces are a relation between time and the power consumption (or RF emanations) at that time. using a Fourier transform1, this can be converted into a relation between various frequencies and their amplitude and phase (which when added together form the original signal2). the side channel leakage, at least for first order attacks, can still show itself in the frequency-domain version of the data, and thus can be extracted that way. the important thing is in the frequency domain we don't actually care about trace offset at all. trace alignment is irrelevant because in the frequency domain, the only difference between two signals that are identical but offset by some amount of time is the phase of the resulting frequency bins (and we ignore phase)
first some probably familiar code if you've seen the previous post. we're using the ASCAD_desync100 dataset and taking 2000 traces out of that (also using the mask metadata)3
ntraces = 2000
db = h5py.File("./ASCAD_databases/ASCAD_desync100.h5", "r")["Attack_traces"]
traces = db["traces"][0:ntraces, :].astype("double")
logger.info("making first order model")
model = numpy.repeat(numpy.arange(256, dtype='uint8'), ntraces).reshape((256, ntraces))
for i in tqdm.trange(ntraces):
(pt, ct, key, mask, desync) = db["metadata"][i]
pt_v = pt[2]
mask_v = mask[15]
model[:, i] = AES_SBOX[model[:, i] ^ pt_v] ^ mask_v
model = numpy_popcount.popcount(model).astype("double").transpose()
ok now the fun part, transform all the traces with scipy.fft
, take the absolute value of the result to discard phase information, and then take the top 50 peaks which we assume contain the leakage4
fft_traces = numpy.zeros(traces.shape, traces.dtype)
for i in range(traces.shape[0]):
fft_traces[i, :] = numpy.abs(scipy.fft.fft(traces[i, :]))
traces = fft_traces[:, 1:350]
avg = numpy.mean(traces, axis=0)
peaks = avg.argsort()[-50:][::-1]
traces = numpy.hstack([traces[:, i:i+1] for i in peaks])
with this, do the typical correlation analysis as before, just like if traces
represented the original time-domain trace set
correlator = Correlator(model)
coefs = correlator.corr_submatrix(traces)
coefs = numpy.abs(coefs)
max_by_key = numpy.max(coefs, axis=1)
plt.plot(max_by_key)
plt.show()
this is a lot more noisy than the time-domain attacks where we took care to align the traces, but the point is we still get the right key byte without any trace alignment being actually necessary, which makes this a really powerful kind of attack
the code (+ previous code) can be found here https://git.lain.faith/haskal/gist/src/branch/gist/sca/ascad/attack.py
approximately. computers aren't able to calculate perfect Fourier transforms (mitigating this requires windowing, which tries to remove the artifacts from the result and there are various kinds of windows you can use with Fourier transforms, but for now ignoring windows is fine)
you might also notice the [1:350]
slice. this is because in this analysis, we discard the 0Hz frequency bin ("DC") because it's not interesting, and the result of running FFT is symmetric so we don't actually need the top half as it mirrors the bottom half
this is similar to the previous post's first order attack. second-order frequency domain attacks require different leakage modelling and it's more complicated. also i haven't written the demo for that yet
an intuitive way to see what a Fourier transform is actually doing is to think about a music visualizer. classic music visualizers have bars corresponding to different frequencies, and the height of each bar indicates how loud that frequency is within the music. visualizers like that are doing Fourier transforms internally to produce that visualization. the important thing is you can do that with any sort of data, not just audio
when you have some crypto algorithms often times you are interested in bonking them, for example by recovering the secret key or making the algorithm accept data that has been tampered with (in traditional parlance, you want to violate properties of Confidentiality, Integrity, and Authenticity). crypto code can have software bugs: if you think about code as a specification of what a program is supposed to do, there can be cases where the specification is simply wrong. but even if the specification is perfectly correct, and would execute flawlessly on an ideal computer, there can still be vulnerabilities in running the code on an actual computer that can still violate important security properties
here's some code
const char* password = getenv("THE_PASSWORD");
bool check_password(char* input) {
if (strlen(input) != strlen(password)) return false;
for (size_t i = 0; i < strlen(password); i++) {
if (input[i] != password[i]) return false;
}
return true;
}
suppose you want to figure out what the password is.... how can you do that? there's nothing wrong with the specification of the algorithm (comparing a password to a correct password -- i mean what could possibly go wrong)
but there's a problem with the implementation
as soon as you get a character wrong it immediately bails and returns false. so a password that has more correct characters takes more time to check than a password that has fewer correct characters. thus you can mount an attack where you guess the first character, and go with the guess that takes the longest to check, then guess the second character, and select the one that takes the longest to check, and so on until you have guessed all the characters
this is what people like to call a timing side channel, and it turns out that you can even do things with timing like guess AES keys by correlating the time it takes to do AES encryptions of random messages to the expected time it would take for cache lookups in the CPU while it's processing the encryption (however, modern CPUs contain hardware AES logic that largely solves the cache timing issue)
ingredients1
the key part is that when the secret combines with your input, this affects some side channel measurement in a way that you can observe. and if you can figure out how that measurement changes with different combinations of your input and the secret, then you have a model for guessing what the secret is. in the timing example, we could develop a model where we said the following
if more characters in our guess prefix match the real secret, then the algorithm will take longer to execute
we reverse that, and now we know that if we measure a longer execution time, that means our input must have been closer to being correct
ok so timing is cool and all but one interesting aspect of actual cryptography on actual hardware, like a microcontroller, is that the microcontroller uses electricity to perform its computations (shocking, i know). more interestingly, the amount of current it draws can be dependent on what exactly it's calculating. correspondingly, it can emit unintentional radio waves as a result of what it's running on the chip as well
these are both some great side channels to use in order to examine what's going on inside a chip when you don't have the capability to decap and microprobe the actual silicon. basically, you can measure power consumption or electromagnetic emissions on a chip, and those signals will contain some information that can be used to recover what the data the chip is processing actually is (like, crypto keys for AES, cause y'know that sounds juicy)
there are two root causes for leakage that are widely used as models that can tell you what the leakage measurement should look like given a certain data value
1
bits in the binary representation of a number. this is because, in silicon, 1 bits are represented by a high signal and 0 bits are represented by a low signal, which means that if a data bus inside the chip is transmitting the data you're interested in, then it's going to use an amount of power for that bus that is proportional to the number of "on" wires in the bus, which is the number of 1
bits in the valuesuppose we have an implementation of AES on a microcontroller and we want to know the secret key used for the AES, and we have some setup that allows us to capture hardware side channel data. this is going to consist of a high speed measurement of the chip's power consumption, or a measurement of an EM probe placed over the chip (ideally near an area of the silicon where you might know your data is being processed -- EM data is also going to need additional filtering and demodulation depending on the exact target hardware and setup). the main component of capturing hardware side channel signals is just a high speed analog-digital-converter of some kind that can record the analog characteristics of the target device (current drawn by the chip, or EM emissions from the chip) very fast. so for capturing this data, people like to use oscilloscopes that can connect to your computer, like a PicoScope. alternatively, the ChipWhisperer is an oscilloscope-like device for side channel education/training
then we have the basic ingredients
more specifically, the AES encryption algorithm has a step in the first round where it combines the key with the input by XOR of the key and the plaintext (AddRoundKey) and then a step where these values are passed through a substitution box (SubBytes). so basically, you can think of AES like this
void AES_encrypt(byte key[16], byte plaintext[16]) {
byte output[16];
// ... some stuff ...
// AddRoundKey
for (size_t i = 0; i < 16; i++) output[i] = key[i] ^ plaintext[i];
// SubBytes
for (size_t i = 0; i < 16; i++) output[i] = sbox(output[i]);
// more steps, and more rounds....
// etc
}
so to develop the model, first we figure out what the value we are targeting is, this should be a value that combines the input and secret (and it's helpful to have a non-linear operation you are targeting, because this reduces the possibility of false positives in your results. the AES sbox is very non-linear). we're going to be attacking each byte of the key individually, so in the following i will assume you have picked a byte to bonk. you can obviously repeat this same attack with the same data for each key byte. here's the model:
for a certain secret key byte and input byte, the code will at some point compute the value
sbox(keybyte ^ inputbyte)
now, we just wrap that in the hardware leakage model (let's say hamming weight): hamming(sbox(keybyte ^ inputbyte))
and yeah that's it, now we have the ability to predict what the leakage should look like given the inputs to the encryption
i assume people generally don't just have a ChipWhisperer on hand to collect some real traces. luckily some good people have prepared a database of AES side channel traces collected on an AVR smartcard platform (it's meant for training machine learning models, but it's also a useful demo dataset)
you can get the data here: https://github.com/ANSSI-FR/ASCAD/tree/master/ATMEGA_AES_v1/ATM_AES_v1_fixed_key
one thing to note is that this is using AES which is masked. masking is a protection against side channel attacks by introducing randomness which is unknown to the attacker on each run of the algorithm, which messes with our ability to predict leakage measurements. so to start, we're going to cheat and use the mask values recorded in the database as if they were known by the attacker, but i will later demonstrate how to bonk the key without knowing the masks
we're going to use python, and you'll need the standard numpy
, scipy
, matplotlib
, and to read the data format, h5py
this can go very fast if you install OpenBLAS. on manjaro, this is the openblas
package (NOT blas
-- that's the reference implementation and it is quite literally 20 times slower)
on debian and friends i believe the package is sudo apt-get install libopenblas-dev
the traces in the ASCAD.h5
database are aligned, which means that every trace (consisting of a series of integers 0-255 reflecting the analog measurements by time) represents starting at exactly the same point relative to the AES operation being performed in the code. when capturing traces, even with very precise triggering to start recording right when the operation you're attacking starts, it can be hard to end up with perfectly aligned traces because of environmental variables outside of your control. so typically some alignment of what you captured is necessary, to make sure every trace's time axis lines up exactly with the rest. for the correlation-based attack here, it's important to make sure the traces are aligned. even 10% of your traces being badly aligned can completely throw off the attack
ASCAD also provides some "desynced" databases, which have been artificially misaligned, which i'll use here to demonstrate trace alignment as in a real attack. to align traces, we can use (FFT) signal convolution (scipy.signal.correlate
) to compute the best alignment for the traces fairly quickly. one issue when doing this with ASCAD which i noticed is that because the traces are very periodic, the best correlation according to scipy.signal.correlate
can be shifted one period sometimes. however, if we take N best correlations and test the Pearson correlation coefficient of each one, then picking the alignment with the highest Pearson correlation seems to get good results
when doing this on real data, it can also be helpful to use the Pearson correlation to throw away garbage traces (ones where the best Pearson correlation is too low, according to some cutoff). you're going to have some of those and you don't want them to be in the data
open the database (we'll use the attack traces section. for our purposes there is no difference between the profiling and attack parts). also, we're only dealing with the first 1000 traces
db = h5py.File("./ASCAD_databases/ASCAD_desync100.h5", "r")["Attack_traces"]
ntraces = 2000
alignments = numpy.zeros(ntraces, "int64")
pick the first trace as a reference trace (make sure to convert it to double. the database stores traces as u8
, which will cause all calculations to wrap and be screwed up)
reference = traces[0].astype("double")
scipy provides correlation_lags
which computes a vector of offsets corresponding to the output of correlate
which can be used to determine which alignment offset ("lag") corresponds with a correlation value. we can use this to shift our traces forwards or backwards to match the best alignment
lags = scipy.signal.correlation_lags(len(reference), len(reference))
now do the alignment on every trace skipping the first one (that has an alignment of 0, since it is the reference). here there is some code that uses a second pass to maximize the Pearson correlation for the top 10 FFT correlation peaks
for i in tqdm.trange(1, ntraces):
trace = traces[i].astype("double")
corr = scipy.signal.correlate(reference, trace)
max_idxs = corr.argsort()[-10:][::-1]
max_corr = 0
max_idx = -1
for idx in max_idxs:
lag = lags[idx]
if lag == 0:
s_ref = reference
s_trs = trace
elif lag < 0:
s_ref = reference[:lag]
s_trs = trace[-lag:]
else:
s_ref = reference[lag:]
s_trs = trace[:-lag]
pcorr = numpy.corrcoef(s_ref, s_trs)[1,0]
if pcorr > max_corr:
max_corr = pcorr
max_idx = idx
lag = lags[max_idx]
alignments[i] = lag
alignments -= numpy.min(alignments)
we can use the alignments to produce a matrix of aligned traces
traces = numpy.zeros((ntraces, 800), 'double')
for i in range(ntraces):
traces[i, alignments[i]:alignments[i]+700] = db["traces"][i]
visualizing is always helpful, so let's plot 10 traces
for i in range(10):
plt.plot(traces[i], color=numpy.hstack((numpy.random.random(3), [0.5])))
plt.show()
you can see here in the central part of the plot every trace lines up very well. most of the traces are layered on top of each other, and each of the peaks are roughly the same shapes. this indicates that we have a consistent time axis with respect to the AES encryption operation happening on each trace
if your dataset contains some garbage (ASCAD, helpfully, doesn't) then there is a chance you will pick a garbage reference trace. you'll notice then that all your correlations will be very low, and in that case you can try picking a different reference trace
now we have aligned traces. somewhere in the trace, at some time point, there is a leakage for our model (recall that the model was hamming(sbox(keybyte ^ inputbyte))
. now since ASCAD has masks we're going to cheat and add in the mask value (for an actual attack, this would be unknown -- stick around), making the model hamming(sbox(keybyte ^ inputbyte) ^ mask)
. (in ASCAD, the mask byte for the sbox output is the last byte of the masks array for an entry in the database)
now... we have the inputbyte
available for each trace. but what about keybyte
? well it's a byte, so it could be from 0 to 255. that's low enough for us to just guess every possible key byte
so we end up with 256 guesses, where guess i
is hamming(sbox(i ^ inputbyte) ^ mask)
let's make that model (because i like yak shaving, i implemented an interface to __builtin_popcount()
in numpy which you can find here. do a pip3 install --user -e .
). here, we're using the 3rd key byte (2, starting at 0) because it's the first masked key byte -- the first two are unmasked and were used for validation -- and we're going to attack the same one later without knowing masks. also, we use the last mask byte which was used in the AES implementation to mask the s-box output
model = numpy.repeat(numpy.arange(256, dtype='uint8'), ntraces).reshape((256, ntraces))
for i in tqdm.trange(ntraces):
(pt, ct, key, mask, desync) = db["metadata"][i]
pt_v = pt[2]
mask_v = mask[15] # cheating here
model[:, i] = AES_SBOX[model[:, i] ^ pt_v] ^ mask_v
model = numpy_popcount.popcount(model).astype("double").transpose()
the model contains a value for each key guess, and for each trace we're analyzing. thus one of the key guesses will very nicely predict the values at a specific tick of the trace set. it's also important to make sure you convert to double
because if you leave the data type as int
then you will have problems in the next steps
so the job is to find which key byte and which tick index are actually matching. one way to do that is to compute the Pearson correlation coefficient between every key guess of your model, and every tick guess of the data. then, get the maximum correlation coefficient, and it should give you the correct key byte and tick index where the leakage occurred2
there's a slow way to do this, just use numpy.corrcoef
using a vstack
of the model and traces sets, and then select a corner of the output matrix which will be the correlation values you're looking for. this is unfortunately more RAM-intensive than we really want -- it outputs a full (ntraces + 256)**2
size square matrix when we really need the 256
by ntraces
corner. because of this i decided to go see how pysca 3 did things and (pysca is fairly unintelligible honestly) found this implementation which i adapted into the following more optimized code, which goes very fast with openblas4
class Correlator:
def __init__(self, P):
self.P = P - numpy.mean(P, axis=0)
self.tmp1 = numpy.einsum("nm,nm->m", self.P, self.P, optimize='optimal')
self.P = self.P.transpose()
def corr_submatrix(self, O):
O -= numpy.mean(O, axis=0)
numerator = self.P @ O
tmp2 = numpy.einsum("nt,nt->t", O, O, optimize='optimal')
denominator = numpy.sqrt(numpy.outer(self.tmp1, tmp2))
return numerator / denominator
this is a class because it remembers the parts of the computation for the model matrix, which allows the corr_submatrix
method to be called multiple times with chunks of data (chunked along the tick axis), and the results concatenated. chunking is useful to avoid eating up tons of RAM, at the cost of performance
now just call it
# uwu
correlator = Correlator(model)
coefs = correlator.corr_submatrix(traces)
coefs = numpy.abs(coefs)
we use the absolute value because when looking for peaks, correlations could be negative (they would not be in this case, but for other types of leakages you might want to catch highly negative correlations as well)
the output is a 256-by-nticks matrix, and the correlation at i,j
represents key guess i
with tick j
collect the max correlations along the key guess axis and plot it
max_by_key = numpy.max(coefs, axis=1)
plt.plot(max_by_key)
plt.show()
yeah. the secret byte is very obvious here, and indeed if you check the x-axis for the peak you'll see it's 0xe0
, which is matches the real 3rd key byte
and in practice you'd just repeat this for all 16 key bytes
ok so now we do it without knowing the mask (a more realistic attack scenario). the problem is, the mask is randomized, so if our model says hamming(sbox(keybyte ^ inputbyte))
, it's not going to match very well with reality, and we're not going to see any peak in the max correlation plot. in fact you can see this if we remove the cheating from the previous code
you can see there is no obvious peak here -- the attack failed
the way to bypass masking is to do what is called a second-order attack. "order" here just means how many points along the time axis you consider at a time. so you might have noticed in the previous section we calculated correlation by a key guess axis and a tick axis, and the result of that was a high correlation value at one specific key guess and one specific tick. in a second-order attack, however, we make combinations of 2 ticks (and an N-order attack would use combinations of N ticks)
to make the combinations of two ticks, we take the absolute difference between the values at both ticks for each trace. this results in a new trace of n*(n-1)/2
values, where each value is the absolute difference between 2 unique values of the original trace
the reason this works is because inevitably with masking, there is going to be some second tick in the traces that is correlated with the mask value. we don't know what the mask value is, but we can use that second tick to subtract the leakage of that mask value from the leakage of the s-box operation (since the state is xor'd with the mask, this corresponds somewhat with a sum of hamming weights, over a large number of random mask values), which would then correlate very well with the unmasked key guesses in the model
first, we'll make a model without cheating
model = numpy.repeat(numpy.arange(256, dtype='uint8'), ntraces).reshape((256, ntraces))
for i in tqdm.trange(ntraces):
(pt, ct, key, mask, desync) = db["metadata"][i]
pt_v = pt[2]
model[:, i] = AES_SBOX[model[:, i] ^ pt_v]
model = numpy_popcount.popcount(model).astype("double").transpose()
and now, make second-order combinations of two ticks as described
nticks = traces.shape[1]
traces_x = numpy.zeros((ntraces, nticks*(nticks - 1)//2), 'double')
tx_i = 0
for i in tqdm.trange(nticks):
for j in range(i+1, nticks):
traces_x[:, tx_i] = numpy.abs(traces[:, i] - traces[:, j])
tx_i += 1
you can see here the absolute difference is being used to generate this new second-order dataset, where the "tick" axis is now a "combination of 2 ticks" axis
now that we have the second order combinations it's the same thing (this time, we split up the data into sequential chunks to lower the RAM usage)
correlator = Correlator(model)
ncolumns = 4
step = traces_x.shape[1] // ncolumns
all_coefs = []
for i in tqdm.trange(ncolumns):
start = i * step
end = (i + 1) * step
coefs = correlator.corr_submatrix(traces_x[:, start:end])
all_coefs.append(coefs)
all_coefs = numpy.hstack(all_coefs)
all_coefs = numpy.abs(all_coefs)
on my computer (skylake laptop, with openblas) this runs in seconds
and now do the same plot along the key axis
max_by_key = numpy.max(all_coefs, axis=1)
plt.plot(max_by_key)
plt.show()
you can see the peak is not as high as before, but it's still very much distinguishable5
one important thing about this is, the more traces you have, the lower the noise floor is going to be. having a low number of traces can result in a plot which doesn't have a clear correct guess on it, even if the leakage is present and correctly analyzed. this becomes important especially when you are analyzing leakage that is not as obviously non-linear as the AES s-box. you can see if you run this second order attack with 1000 traces instead of 2000, the peak will be much less obvious (and barely the maximum). and if you run it with 3000, the peak will become more obvious
the other thing to realize is that the correlation can reveal correct secret values if you have literally any advantage in your model over the noise floor. if you can guess leakage in your model better than pure randomness even just sometimes, then with enough traces you'll be able to leak the secret. the correlation attack is extremely powerful and very fast. even for a (quadratically scaling) second-order combination of ticks in your trace data
the full code is available at https://git.lain.faith/haskal/gist/src/branch/gist/sca/ascad
now, go hack yourself some AES keys for great good
or other stuff. i wanted to develop a from-scratch HMAC-SHA256 attack and maybe XChaCha20 too but i haven't gotten to it yet (these are more involved because they require more leakage steps of more linear operations -- like addition and xor, which are hard to deal with)
i would like to extend a big thanks to Taowa for providing helpful feedback on this post
we're going to focus on non-profiled side channel attacks, which means you just get a trace set with varying inputs and you have to find the secret. there's also a class of profiled side channel attacks, which have the capability for the attacker to use a copy of the system to perform analysis on traces where the secret and input are both known, and then use that information to attack the attack system where the secret is unknown. most importantly, attacks based on deep learning (that's where the ASCAD database we will use comes from)
there are other ways to do this, like traditional differential power analysis, which bins the traces based on bits, or mutual information analysis, which just uses another statistical measure (mutual information, as you might guess). but here we're going to use the correlation method
an influence/precursor of the better known jlsca
pysca is available under a GPL-3.0 license which you can find in the repo. my implementation is also licensed under GPL-3.0
the reason it's not as high is because the assumption we made that XOR operations correspond with a sum of hamming weights and thus subtracting hamming weights is sufficient to unmask a leakage value is not exactly correct. but over a large number of random inputs it will be good enough that there is still some correlation we can observe
haskal what's the point of this
sometimes when people ask how i got good at computer i am like, well, you just need to get good at duck duck going and that's basically it. idk how to do anything i'm just good at typing queries into duckduckgo tbh
the point is you don't have to be "smart". the whole thing is computers isn't about knowing at all. it's about learning (often times by literally internet searches). you have to be willing to learn. that's all you need
what i'm annoyed by is that a lot of the time people like to talk about "intelligence" or "IQ" as some sort of fundamental property of people that gates their ability to do things and i'm here to tell you that's pseudoscience and also fake. "IQ" doesn't meaningfully predict literally anything (for example... how well you might do in school. your zip code predicts that better than "IQ". which is honestly a really sad thing to me -- the thing is that a lot of the time "intelligence" that people think about is actually just a measure of whether you are visibly lower-class and/or disabled). problem is a lot of the time you'll see techbros on about some "intelligence"-related bullshit. or the whole concept of a "10xer", which is so patently bullshit it hurts. and what also sucks is whenever i see people gatekeeping with this absolute pseudoscience, which i'll also mention has concerningly been associated with racism and eugenics. i can pretty much guarantee that your "IQ" is not a predictor of your programming ability, except specifically the extent that thinking you have a "low IQ" might discourage you from trying
straight up, if you read How to Design Programs and do the exercises you'll be on par with literal senior engineers in this fucking industry. i'm not kidding
and with a solid base in the basics you can get away with almost anything using only just-in-time knowledge acquisition (duckduckgo)
anyway, i think "intelligence" is worthless, basically. if you go to an american school these days they might tell you about what they call a "growth mindset" focused on describing your brain as a muscle that can be trained, not just stuck with some static predetermined amount of "intelligence" and tbh basically that. i think that kinda like getting physically fit, the more stuff you learn the easier it gets. the reason computersโข is a field i got into is just because a lot of the information is readily available online, and the materials you need to interact with this field is like, A Computer, which makes the barrier to entry really low compared to literally most other things. and i think that's good
fundamentally there's no special sauce to, for example learning everything that i know about on any topic. i maintain that anyone can do it if you go through the same materials as i did. anyone who tells you "intelligence" matters is a cop and should not be trusted imo. people who tell you they have "high IQ" or are a "10x rockstar" are to be ignored. you can learn whatever the heck you want to, i promise
]]>map
and sub_filter
(which are stock in the debian distribution of nginx)
the basic idea is: match on the current millisecond, and map that to your random content string. then, put a sentinel in your HTML to get replaced with the random string (here it's RANDOM_TEXT_SUB
). finally, use sub_filter to do the replacement
here's how
# in http {
map $msec $rnd_text {
default "";
~0$ "message 0";
~1$ "message 1";
~2$ "message 2";
~3$ "message 3";
~4$ "message 4";
~5$ "message 5";
~6$ "message 6";
~7$ "message 7";
~8$ "message 8";
~9$ "message 9";
}
server {
# ....
location /motd.html {
sub_filter RANDOM_TEXT_SUB $rnd_text;
}
}
if you have a number of random messages that isn't 10, you can match arbitrarily on ranges of trailing digits using regex (that's what the ~
does, it's a regex match). For example, for 3 messages you can use
~[0-3]$ "first message";
~[4-6]$ "second message";
~[7-9]$ "third message";
this isn't perfectly split, but it's good enough. you can use more trailing digits for more accuracy.
that's basically it. you can see a demo at https://joan.awoo.systems/random.html
you can also use this for different types of basic dynamic content. for example, once i was playing with nginx-rtmp and dash streaming, and wanted to provide a useful fallback for people with javascript disabled (which is, imo, a perfectly reasonably thing to want). so the noscript tag on this page says something along the lines of "you can view this stream in a regular video player at rtmp://...the url..."
now the problem for this was... this is a static html page, how do we insert the url especially when the context is specifically a lack of javascript? well, i used the same thing, with sub_filter inserting a url based on the request path, and it worked :P
]]>you can bind the empty identifier. no, really
the pipe |
is the escape character for symbols, so eg |5|
is the symbol 5, rather than the number 5
so what happens if we use ||
?
$ racket
Welcome to Racket v7.9 [bc].
> (define || "meow")
> (displayln ||)
meow
๐ค
> (namespace-variable-value (string->symbol ""))
"meow"
yeah.
anyway
my current bit is going to be swapping the title and subtitle of posts, just for fun
i wanted to demo a cool thing you can do when you're too lazy to solve logic puzzles. the example here is a puzzle "icebreaker" from the previous teammate puzzle hunt: https://teammatehunt.com/puzzles/icebreaker
spoilers follow
.
..
...
....
here's the puzzle contents
MARTIN: "Before the party, letโs play a quick icebreaker. Iโll start. My three statements are:
- The five of us are, in no particular order, the person who lives in the first house, RAMSEY, the person who has a pet FERRET, the person who has a pet POODLE, and the person from KANSAS.
- TURNER and the person who owns WHALES live on opposite ends of the street.
- WALTER is from FRANCE, but doesnโt own a FERRET."
RAMSEY: "Iโll go next. My three statements are:
- MARTIN and the person from LONDON live on opposite ends of the street.
- SANDRA is from LONDON, but doesnโt own a FERRET.
- WALTER lives immediately to the left of me."
SANDRA: "Iโll go next. My three statements are:
- I own pet ROBINS.
- The person who owns WHALES lives two houses away from the person from FRANCE.
- WALTER is from TAIPEI."
TURNER: "Iโll go next. My three statements are:
- I am not from KANSAS.
- I was alone yesterday taking my TIGERS to the vet.
- RAMSEYโs pet ROBINS live three houses away from me.
WALTER: "Iโll go last. My three statements are:
- I met up yesterday with the person who has a POODLE and also the person who lives in the second house.
- The five of us are, in no particular order, me, SANDRA, the person who owns ROBINS, the person from MUMBAI, and the person from TAIPEI.
- TURNER is from KANSAS, and doesnโt live in the first house.
this is a bog standard logic puzzle and given the title "icebreaker" you can convince yourself that the theme of this icebreaker is "two truths and a lie" which implies that one of each set of 3 statements is false, and the other 2 are true
if you're familiar with this sort of thing you may be going straight to google sheets and making a logic grid but we can do better than that because it turns out that people have already made cool programs that automate the process of solving satisfiability problems so if we can convert all of this text into a giant boolean expression then feed it to a solver it should give us answers... and hopefully not unsat ...
to be honest my initial implementation of this problem was entirely booleans, which meant the code ended up being very large and very spaghetti and contained some very spaghet manual axioms that were necessary in the system before it arrived at a working solution. so i rewrote it, because it turns out there's a better way to represent the problem
we have the following categories of things
there are five items in each category, and the statements in the puzzle text provide some expressions over these that might be true or false
the representation i chose for a cleaner solution was
this means we get basic axioms for free from integers and don't have to define them ourselves, which is a pain
anyway here's the setup
#lang curly-fn rosette
(require syntax/parse/define
(for-syntax racket/syntax))
oh yeah we're going to be using rosette, a solver aided programming library for racket, just because we can
and we're using macros. you bet
;; house is represented by actual number
;; people, places, pets represented by symbolic variable
(define-for-syntax *data*
'((name martin ramsey sandra turner walter)
(place london kansas france taipei mumbai)
(pet robins tigers whales ferret poodle)))
;; create a solver and define all the symbolic variables
(define-simple-macro (define-symbols/solver solver:id)
;; generate list of ids, bound to calling lexical context, with a syntax prop determining the
;; category
#:with (ids ...) (for*/list ([row (in-list *data*)] [val (in-list (rest row))])
(syntax-property (format-id #'solver "~a" val) 'icebreaker:cat (first row)))
;; create the symbolic define expressions
#:with (defs ...) (for/list ([id (in-list (syntax-e #'(ids ...)))])
#`(define-symbolic #,id integer?))
;; create initial constraints
#:with (stmts ...) (let ([groups (group-by #{syntax-property % 'icebreaker:cat}
(syntax-e #'(ids ...)))])
(append (for*/list ([grp (in-list groups)] [id (in-list grp)])
;; all in [1, 5]
#`(void (solver (<= 1 #,id 5))))
(for/list ([grp (in-list groups)])
;; no two in the same category are equal
#`(void (solver (mutually-exclusive #,@grp))))))
(begin
;; incremental solver
(define solver (solve+))
defs ...
stmts ...))
(define-symbols/solver inc)
this is the bulk of the setup. it generates all the variables and creates a new SMT solver by macro and helpfully adds some basic assertions to the solver: each symbolic variable should be 1-5 and no two variables in the same category should be equal (mutually-exclusive
is a custom macro, that's coming up it's not that complicated)
;; helper for defining the statements of one person (2 true, 1 lie)
(define-simple-macro (define-statements solver:expr sfirst:expr ssecond:expr sthird:expr)
(void
(solver (or (and sfirst ssecond (not sthird))
(and sfirst (not ssecond) sthird)
(and (not sfirst) ssecond sthird)))))
this is a macro that defines a set of 2 truths and a lie made by a person in the puzzle text. it basically just expands the set of three statements to assert that there is some combination where one is false and two are true
;; helper for defining a mutually exclusive set of symbolic and concrete values
(define-simple-macro (mutually-exclusive options:expr ...)
#:with (stxs ...) (for/list ([combo (in-combinations (syntax-e #'(options ...)) 2)])
#`(not (= #,(first combo) #,(second combo))))
(and stxs ...))
;; helper for asserting that 2 houses are at a certain distance apart
(define-simple-macro (at-distance v1:expr v2:expr dist:expr)
(= dist (abs (- v1 v2))))
this is the last bit of setup. there's the mutually-exclusive
macro and a macro for defining that 2 houses are at a certain distance apart (this happens a few times)
now we make the assertions, transcribed from the text
;; the rules
;; martin
(define-statements inc
The five of us are, in no particular order, the person who lives in the first house, RAMSEY, the person who has a pet FERRET, the person who has a pet POODLE, and the person from KANSAS.
use the handy mutually-exclusive
macro
(mutually-exclusive 1 ramsey ferret poodle kansas)
TURNER and the person who owns WHALES live on opposite ends of the street.
"opposite ends" is the same thing as distance 4 apart, so we use that macro. also this asserts that the two people here are distinct, so that's another thing to add
(and (at-distance turner whales 4) (not (= turner whales)))
WALTER is from FRANCE, but doesnโt own a FERRET.
this is relatively basic
(and (= walter france) (not (= walter ferret))))
and so on for the rest of the statements
;; ramsey
(define-statements inc
(and (at-distance martin london 4) (not (= martin london)))
(and (= sandra london) (not (= sandra ferret)))
the next statement is about being "immediately to the left". we assume house numbers go left-to-right (this turns out to be correct) and thus walter's number is ramsey's minus 1
(= walter (sub1 ramsey)))
;; sandra
(define-statements inc
(= sandra robins)
(at-distance whales france 2)
(= walter taipei))
;; turner
(define-statements inc
(not (= turner kansas))
(= turner tigers)
(and (= ramsey robins) (at-distance ramsey turner 3)))
;; walter
(define-statements inc
this next one is also a case for mutually-exclusive, where the speaker, the poodle, and 2 are mutually exclusive (if you read the statement carefully this makes sense)
(mutually-exclusive walter poodle 2)
almost done..
(mutually-exclusive walter sandra robins mumbai taipei)
(and (= turner kansas) (not (= turner 1))))
whew that was a lot of statements
finally we print the answer (hopefully it's sat!!)
;; this increments the solver with trivial truth because that's what we need to get the
;; model out
;; the define-statement macro voids everything so you can't get the last result normally
;; whatever. don't yell at me pls
(define solution (inc #t))
(displayln solution)
and this is what it outputs
$ racket icebreaker.rkt
(model
[martin 3]
[ramsey 2]
[sandra 4]
[turner 5]
[walter 1]
[london 4]
[kansas 5]
[france 3]
[taipei 1]
[mumbai 2]
[robins 2]
[tigers 5]
[whales 1]
[ferret 3]
[poodle 4])
and there you go. logic puzzle solved. this lists the house numbers for each name, pet, and place where person is from
if you evaluate the original statements with this solution, you'll find that the following end up true (and the rest false)
the solution checks out
(there's a final step to get the actual answer to this puzzle but i'm not going over it because it's not super relevant. exercise to the reader, or something,)
you can find the full code (concatenation of all these blocks) here: https://git.lain.faith/haskal/gist/src/branch/gist/racket/icebreaker/icebreaker.rkt
haha no, but the solution is still interesting imo
manually solving this is probably a lot faster than writing out this SMT solver code and fixing the bugs if it's unsat or underconstrained. but there's of course also overhead in the macros and such for the code here that, while they took a bit to initially write here, could probably be reused for other einstein-type puzzles. maybe making a full logic puzzle assistance program based on SMT could be useful for future puzzlers ๐๐ฆ
tune in next week for when i hopefully write another blog post because i rly should be writing stuff more than like, once every 2 months tbh
]]>