Blake Miner's Blog

Late in 2016, I began development on a lightweight, isomorphic WebSocket library for Node.js called ws-wrapper .  Today, this library is stable and has been successfully used in many production apps. Why?  What about socket.io ?  In my opinion, socket.io and its dependencies are way too heavy .  Now that the year is 2018, this couldn't be more true.  Modern browsers have native WebSocket support meaning that all of the transports built into the socket.io project are just dead weight.  On the other hand, ws-wrapper and its dependencies weigh about 3 KB when minified and gzipped.  Similarly, ws-wrapper consists of about 500 lines of code; whereas, socket.io consists of thousands of lines of code.  As Dijkstra once famously said: "Simplicity is prerequisite for reliability." ws-wrapper also provides a few more features out of the box.  The API exposes a two-way, Promise-based request/response interface.  That is, clients can request dat...

This blog post will cover the varying levels of data caching and data persistence provided by various operating systems and database systems. Cache  - A device placed in front of another storage device that is used for temporary storage of data.  The use of a hardware cache is often transparent to the end user and is generally smaller, faster, and less persistent than the backed storage device.  Rather than reading/writing from/to a slower storage device (i.e. a hard disk), we read/write from/to the cache (i.e. RAM, NVRAM, etc.). Reading from a cache is simple.  If the datum we requested is in the cache, we simply read it from the cache.  If not, we need to retrieve the datum from the storage device and place it into the cache (often a much slower operation). There are different policies when we write to the cache, though... Write-Through Cache Policy  - When using this policy, writing over the cache also writes over the data on the backed storage d...

The purpose of this blog post is for me to ramble about Meteor, a new open-source platform for building JavaScript web applications in Node.js. Likes: JavaScript (Node.js) - If you know me, you know that I love JavaScript! Live Binding / Live Page Updates - IMHO, this is the best feature in Meteor.  View templates update automatically when data in your model changes.  Very cool! Data Synchronization - I like that the client and server communicate and synchronize data, but having database APIs on the client might not be the way to go... Latency Compensation - Cool Hot Code Pushes - A shiny new toy.  It's cool, but I'm not quite sure how useful this will be to most developers. Dislikes: Database APIs on the client - Okay, yes... having access to your database from any web browser can be nice for programming, but it makes it that much easier for a malicious person to compromise sensitive data.  If someone gains unauthorized access to your system (i.e. som...

As a web developer, generally speaking, you should consider supporting the following browsers (at the time of this writing): Chrome (latest) - the browser that sets the bar for the others; you should be using it and supporting it Internet Explorer 9+ - the browser that finally caught up with the times a bit; basically, a Chrome wannabe.  I still say that IE sucks... even if it really doesn't anymore.  Yes... I'm sour about IE8 and below. Internet Explorer 8 - the old, sad browser that we sadly still have to support for a while.  CSS 3 is not well-supported here, so we use projects like CSS3 PIE or whatever.  By the way... IE8 sucks.  I can't wait until this comes off of the list. Firefox (latest) - the browser that was once awesome and has sadly suffered recently because it's slower than Chrome... but hey, lots of people still use it. Safari (latest) - Watch out for Safari as more iPhones, iPads, Macs, and more overly-priced Apple products flood the ...

Two nights ago, a leap second was added to system clocks running Linux, causing much-undesired havoc. On July 1st at 12:00 AM UTC, both of my Amazon EC2 instances fired an alarm indicating high CPU usage. I investigated to find that it was MySQL that was eating all of the CPU. I logged in and ran SHOW PROCESSLIST to find that no queries were running (these servers don't get hit much after business hours). I stopped MySQL, CPU utilization dropped back down to 1-3% (as normal). I restarted MySQL, and it started eating a lot of CPU again. Then, I restarted the server (shutdown -r now), and the problem went away. Both servers had the exact same problem (running Ubuntu 12.04 LTS). In my particular case, MySQL began eating CPU, even after being restarted.  It was a livelock. The only relevant item I saw in the syslog was: Jun 30 23:59:59 hostname kernel: [14152976.187987] Clock: inserting leap second 23:59:60 UTC Oh yeah... leap seconds.  Those are super impor...

One thing I never really liked about JavaScript was the way it treats null , undefined , and false .  It's just, well... weird. Another example of this that I discovered today... null is greater than or equal to zero, but it's not greater than 0, nor is it equal to zero.  WTF? null > 0 false null >= 0 true null == 0 false

Carla and I are planning on getting married sometime in October next year.  We need to pick a date, and that decision may  involve some science and mathematics.  :) For example, we want the weather to be nice.  To be more precise, we'd like the high temperature for the wedding day to be between 60 and 80 degrees Fahrenheit.  Obviously, we have both lived in Ohio our entire lives, and we have a pretty good idea of what the weather will be like.  We both hypothesised that October was a "hit or miss" sort of month; it could be cold, or it could be nice. But, for me, a simple hypothesis was not enough; I really wanted to know the probabilities of decent weather based on historical weather data.  Many websites on the Internet (i.e. almanac.com) charge you to review historical weather data, but Carla and I discovered a cool page on cleveland.com that provided exactly what we wanted.  I loaded the historical temperature data from 1903 to 2011 f...

http://xkcd.com/844/

Email sucks, and it's incredibly broken! I mean.... sending an email in Gmail is obviously quite trivial.  But working with SMTP and email servers... not so much. Most people take email for granted.  It was invented way before I was born... way before Windows 95, Windows 3.1, or even DOS!  What?!?! Yeah... so not surprisingly, email sucks.  Remember... DOS sucked and Windows 95 sucked (not at that time, of course).  In fact, email sucks so much, it makes me sad that we still use it.  It's almost embarrassing... almost as embarrassing as Internet Explorer. Don't believe me?  Just go to the Postfix mail server documentation page.  Their website is straight out of the late-80's.  I think there might be one  image on the entire website?  After all, we can't have those 4.8K modems downloading images... it will take too long, tie up your phone line, and waste too much bandwidth.  :) So, why am I complaining about email? ...

I'm pretty sure that if I've played 100 games and won exactly 1 of them, my winning percentage should be 1%. I might be wrong, though.  There is no rounding error when you do 1/100 * 100%.  Single-precision floating point number rounding error?  For some reason, I doubt it... Weird... and stupid... EDIT:  Wrote a C program to make sure that Microsoft's Minesweeper has a bug. $ cat main.c #include <stdio.h> int main() {         float x = 1.0 / 100 * 100;         printf("%f\n", x);         return 0; } $ gcc main.c; ./a.out 1.000000 ... Yep... sure does.

I recently asked  this question on StackOverflow  about how to join to a derived table using an index.  I found out that even if your derived table is generated with tables that use all of the proper indexes, and you try to join that derived table to a base table in your outer query, no indexes are used.  Ever... So, let's start with a contrived example (as we often do)... SELECT * FROM t1 JOIN (SELECT * FROM t2) AS derived_t2 ON t1.f1=derived_t2.f1; OK... so let me get this straight... even if t1.f1 and  t2.f1  are indexed, the join will still be a full table scan?  You betcha! Let me say that again...  whenever you write a subquery and try to join the derived table to your outer query tables, no indexes are ever used .  It's a full table scan. So, I think to myself... "Wow... so that's really stupid, actually... because I have tables that only have a few thousand rows in them, and the query performance is shit.  Like... ...

If you're like me, then you are probably building web applications using Node.JS and MySQL (and maybe Redis, too).  If so, you're probably going to need transactions, and you've probably already noticed that the current version of node-mysql doesn't support transactions yet.  :( But that's OK because I have a solution for you.  Check out node-mysql-queues on github .  This project provides pretty good support for MySQL transactions with a fairly simple API.  There are a couple of things to remember, though.  For one, Node.JS is very "callback-centric," so when executing a series of queries, you would normally chain the queries together with a series of callbacks.  node-mysql sort of changes this model, by allowing you to place queries on a queue to be executed in order.  If you only care about doing something when all of your queries are done, you can simply put your callback in the final query.  node-mysql-queues allows you to do the same...

Basically this blog post is about dynamic columns in MySQL queries.  Since you can't run "pivot" queries in MySQL, how do you write queries that return an arbitrary number of columns (perhaps based on the number of rows in another table)? The answer is simple: you can't... at least not in a single query.  The best way to do this is to run two queries and generate the SQL for the 2nd query based on the results of the 1st query. Imagine this table of product sales: ProductID CustomerID Unit Sales 1 1 20 1 2 50 2 1 20 2 2 70 2 3 90 Maybe you want your result set to look like this... ProductID Customer 1 Customer 2 Customer 3 1 20 50 0 2 20 70 90 Write this query first: SELECT CustomerID, CustomerName FROM `customers` Store the results into $rows.  For each $row, append this string to $fields: ", SUM(CASE WHEN CustomerID=$row[0] THEN `Unit Sales` ELSE 0 END) AS $row[1]" Then... write this query: "SELECT ProductID ...

I am so frustrated right now that I am going to dedicate an entire blog post to how crappy Internet Explorer is. With all due respect, Microsoft, IE is terrible. Thank you for your understanding. Sincerely, The Management

This article is for those of us who have heard all of the buzz about Node.JS and they need to convince their boss or their clients to make the switch.  Or, perhaps... you're not quite convinced yet? Why Node.JS? Reason #1 - JavaScript is awesome.  So, in case you didn't know, JavaScript is awesome.  I'll let you read my other blog post  for more details, but I'll also summarize for you: The language itself is extremely powerful.  JavaScript comes packed with modern programming language features like closures, dynamic data types, first-class and higher-order functions, prototypes (not classes), lack of concurrency  (more on this later), exception handling, garbage collection, and much, much more!  These features help make your code better, keep you from making mistakes, and make your applications more bulletproof. JavaScript is widely adopted.  It's the scripting language of the web.  Need I say more?  Almost any device with a web...

For-In loops in JavaScript are really  nice... until they aren't. Not knowing exactly  how they work might bite you.  I'll start with an example: var arr1 = ["a", "b", "c", "d"]; for(var i in arr1)     console.log(i, arr1[i], i+1, arr1[i+1]); Seems simple enough.  What will be printed? 0 a 01 undefined 1 b 11 undefined 2 c 21 undefined 3 d 31 undefined  Whoops... I am an idiot!  That's not exactly what I wanted.  Note that i is a string , not a number .  I wasted an hour of my life trying to figure out why i+1 was not working properly... only to come to this horribly obvious conclusion.  It was totally worth a blog post... and a beer. Sometimes the old-fashioned for loops are better for Arrays: var arr1 = ["a", "b", "c", "d"]; for(var i = 0; i < arr1.length; i++)     console.log(i, arr1[i], i+1, arr1[i+1]);

Q: What's ExpressJS? A: quite possibly the coolest thing to happen to Node.JS... EVAR!  This is awesome news for web developers and programmers. OK, you're interested now.  So, if you've followed my blog recently, you know that JavaScript is pure awesomeness.  You also know that server-side JavaScript is the way of the future.  Node.JS is the server-side JavaScript environment.  ExpressJS is the library. Q: So, what do you currently do to write web applications and web services? A: I write PHP code that runs on an Apache web server (or Lighttpd-FastCGI if you prefer).  The PHP code does cool stuff like session handling, parsing URL-encoded POST responses, talking to a database (probably MySQL), generating HTML, etc.  I use HTML, CSS, and JavaScript for the client-side stuff. Q: What do you REALLY want to do? A: Uhhh.... write awesome JavaScript code??? YES!  That's exactly right! You don't need PHP anymore! (Wha???)  You don't ev...

What are closures in JavaScript? Taken from  http://jibbering.com/faq/notes/closures/ : The simple explanation of a Closure is that ECMAScript allows inner functions; function definitions and function expressions that are inside the function bodes of other functions. And that those inner functions are allowed access to all of the local variables, parameters and declared inner functions within their outer function(s). A closure is formed when one of those inner functions is made accessible outside of the function in which it was contained, so that it may be executed after the outer function has returned. At which point it still has access to the local variables, parameters and inner function declarations of its outer function. Those local variables, parameter and function declarations (initially) have the values that they had when the outer function returned and may be interacted with by the inner function. For Example: var x; function foo() {     var i = 2; ...

Here's a bold statement... (so I made the font bold) JavaScript is the best programming language Here's why... The language is extremely powerful .  Let's imagine that we are inventing a new programming language.  The most important thing we will focus on is "power."  That means... write more elegant , more flexible , more readable programs... and write them  even faster . Closures.  A completely awesome, overlooked feature of JavaScript that we all (hopefully) take for granted.  Closures allow you to create a new instance of a function which retains access to the context in which it was created.  Examples are here:  http://en.wikipedia.org/wiki/Closure_(computer_science) . Dynamic data types.  JavaScript variables aren't restricted to any particular datatype.  Store a String, store a number, or store an Object.  JavaScript automatically casts datatypes back and forth as needed. Objects are basically associ...

I found this list absolutely hilarious.  I've starred the ones that I use the most.  :) Top 25 Explanations by Programmers when their programs don't work: 1. Strange...  **** 2. I've never heard about that. 3. It did work yesterday. ** 4. Well, the program needs some fixing. 5. How is this possible? ****** 6. The machine seems to be broken. 7. Has the operating system been updated? 8. The user has made an error again. **** 9. There is something wrong in your test data. 10. I have not touched that module! 11. Yes yes, it will be ready in time. ** 12. You must have the wrong executable. 13. Oh, it's just a feature. ** 14. I'm almost ready. ** 15. Of course, I just have to do these small fixes. **** 16. It will be done in no time at all. *** 17. It's just some unlucky coincidence. 18. I can't test everything! 19. THIS can't do THAT. 20. Didn't I fix it already? ****** 21. It's already there, but it has not been tested. 22. I...