You might think that data com­pres­sion is a solved problem, lossless or lossy. But, no. Gzip and related formats like zlib, Zip, and PNG that use the DEFLATE algorithm were great 25 years ago, still do a decent job, and are completely ubiquitous and in­dis­pens­able, but there are better, smarter algorithms now.

Google has announced two new com­pres­sion formats in the last couple of years, Zopfli and Brotli. Zopfli does a better job of generating Deflate-compatible data, although it's very slow. Brotli gives ~20% better com­pres­sion than Deflate, but at about the same speed. Then there's xz, which grew out of 7-zip, and also works well.

Zstandard has just been announced …continue.

Thanks to Tom Limoncelli, I became acutely aware of USB charge-only cables and condoms. If you plug your phone into an unknown computer to charge the battery, you run the risk of having your phone hijacked by malware. USB transfers data as well as elec­tric­i­ty and you're es­sen­tial­ly giving the computer un­re­strict­ed access to your phone.

Certain USB cables are charge-only and will not pass data. There are also “USB condoms”, which are inserted between the cable and the computer. They not only block data, but they can po­ten­tial­ly charge the battery faster, as they can switch the device into a fast-charging mode. I've ordered a pair from Amazon, as we're …continue.

During an internal training exercise today, as a sort of one-man Chaos Monkey, I de­lib­er­ate­ly broke a test system by changing a config setting to read:

itemfinder.url = http://test-іtemfinder.example.com/

The correct value should have been:

itemfinder.url = http://test-itemfinder.example.com/

What's that, you say? There's no difference, you say?

There is a difference, but it's subtle. The first i in the URL is 'CYRILLIC SMALL LETTER BYELORUSS­IAN-UKRAINIAN I' (U+0456), not 'LATIN SMALL LETTER I' (U+0069). Depending upon the font, the two is may be visually in­dis­tin­guish­able, very similar looking, or the Cyrillic i may not render.

This is an example of an In­ter­na­tion­al Domain Name Homograph Attack. There are Greek letters and Cyrillic letters that look …continue.

I was in­ves­ti­gat­ing the per­for­mance of a web app today, and I spent some time looking at the Flame Chart vi­su­al­iza­tion in Chrome's profiling tools, which helped identify some problems.

Flame Charts are like Brendan Gregg's Flame Graphs, except that the charts are sorted by time, while the graphs are sorted al­pha­bet­i­cal­ly.

Quoting from Gregg's recent ACM Queue article:

A flame graph has the following char­ac­ter­is­tics:

• A stack trace is rep­re­sent­ed as a column of boxes, where each box represents a function (a stack frame).
• The y-axis shows the stack depth, ordered from root at the bottom to leaf at the top. The top box shows the function that was on-CPU when the …continue.

I recently learned from a Stack­Over­flow question that the rounding behavior in Python 3.x is different from Python 2.x:

The round() function rounding strategy and return type have changed. Exact halfway cases are now rounded to the nearest even result instead of away from zero. (For example, round(2.5) now returns 2 rather than 3.)

The “away from zero” rounding strategy is the one that most of us learned at school. The “nearest even” strategy is also known as “banker’s rounding”.

There are actually five rounding strategies defined in IEEE 754:

Further …continue.

I learned today about the -v (--verbose) flag to git commit (git-commit), which causes a unified diff of what would be committed to be appended to the end of the commit message. This diff is not part of the commit. Set the commit.verbose con­fig­u­ra­tion variable (new in Git 2.9) to adjust the default behavior.

I also learned about using git show (git-show) to display the diff for the most recent commit. I had been using git log -1 --patch (git-log). More on git log -p vs. git show vs. git diff.

I needed to create a wildcard SSL cer­tifi­cate and upload it to AWS CloudFront today.

First, generate a 2048-bit private key. This will prompt you for a passphrase:

$ openssl genrsa -des3 -out example.key 2048

Check which signature algorithm was used (SHA-256 is rec­om­mend­ed):

$ openssl req -in example.csr -noout -text

Transform the private key to PEM format:

$ openssl rsa -outform PEM -in example.key -out example.pem

Generate a Cer­tifi­cate Signing Request. Note the * in the server FQDN:

$ openssl req -new -key example.key -out example.csr

Country Name (2 letter code) [AU]:US
State or Province Name (full name) [Some-State]:Washington
Locality Name (eg, city) []:Seattle
Organization Name (eg, company) [Internet Widgits Pty 

I ac­ci­den­tal­ly deleted a note that I shouldn't have in Evernote. It wasn't obvious how to undelete it. I had to resort to the help:

• Open Notebooks
• Scroll all the way to the bottom
• Open the Trash notebook, which is unlabeled in the Mac desktop version of Evernote
• Restore the deleted Note

I ran a script this afternoon that died mys­te­ri­ous­ly without any output. It was using SQLAlchemy to query all the rows from a large table so that they could be trans­formed into JSON Lines to be loaded into Elas­tic­search. When I reran my script, I noticed this time that something had printed Killed at the very end.

A little research convinced me that the OOM Killer was the likely assassin. I looked in /var/log/kern.log and I found that my process had used up almost all of the 8GB on this system before being killed.

The query had to be the problem. A little more research led me to augment my …continue.

I've had to figure this out twice in recent months, and it was no easier the second time than the first.

If you reinstall the Facebook app on Android, you will be plagued by the phone buzzing every few minutes to notify you that someone posted something.

The relevant setting is buried deeply.

• Open the Facebook app
• Click the gray-on-white hamburger icon. Top right, second row.
• Scroll all the way down to App Settings. Click.
• Click No­ti­fi­ca­tions. It's not obvious that it's clickable.
• Change the Vibrate setting to OFF
• Curse Facebook.