Exactly Once Execution In A Distributed System

2017 Sep05

S

kyring is a distributed system for managing timers, or delayed execution similar to setTimeout in javascript. The difference being that it is handled in a reliable and fault tolerant way. setTimeout in javascript is transient. If the running application is restarted or crashes, any pending timers are lost forever. The one guarantee that skyring provides is that a timer will execute after the specified delay, and that it only executes once. Exactly once is an interesting challenge in distributed systems, and Skyring makes use of a number of mechanisms at the node level to achieve this. From a high level, this is what the behavior on individual nodes looks like.

Skyring Node Behavior

Shared Nothing

Skyring follows the shared nothing mantra similar

Read More

Distributed Timers With Node.js and Skyring

2016 Dec28

W


orking with timers a distributed system is a really nasty problem that pops up more often than most people would like. Something as simple an useful as setTimeout / clearTimeout becomes brittle, unreliable and a bottle neck in today's stateless scaleable server mindset. Basically, I need to be able to set a timer somewhere in a cluster with out knowing or caring about what server. And reversely, I need to be able to cancel that timer without having to know where in the cluster that timer lives. But before we can start to understand possible solutions, let's dive into a use case to understand the problem and why existing solutions aren't suitable replacements.

Scenrios

Un-send an email - A simple

Read More

Ubiquitous Fine Grained Access Control With Node.js and RethinkDB

2016 May31

A

cess control is apart of virtually every application. Certain users should be able to see / do X, but never be able to do Y. Usually these kind of requirements are expressed in overly simple terms, such as:

Admin users should see that button but regular users should not.
--Every Product Manager. Ever

With traditional RDBM Systems, the direct approach is to set up a Many-To-Many relation ship between Users and Roles Where roles are basically like tags. Subsequently code paths are created to check if a user has a role named admin . Which gets very ugly as these sorts of simple checks quickly have to cover increasingly complex logic for users with multiple roles and for roles that

Read More
filed under:  javascript rethinkdb acl node

ES6 Generators and Iterators

2015 Mar03
N

ode.js ( and more recently io.js ) has put javascript as a programming language on a fast path. We have seen more improvements and advancement in the last 2 years that we have seen in the previous 2 decades. ECMAScript version 6 ( ES6 ) or `harmony` bring a lot of interesting and very useful features. One of the more useful, and oddly enough, confusing features is the introduction of generators and iterators.

The idea of generators and iterators are not unique to javascript. In fact that are fairly common in other programming languages. The Python language is very well know for it wide support for generator / iterator support. In fact it ships with module dedicated to creating and dealing with

Read More

Better Errors Through JavaScript Inheritance

2015 Jan24

O

ne of the more frustrating things in JavaScript has always been how Errors are handled. They are difficult to check for, harder to extend and often times have cryptic messages providing little insight into what has gone wrong.

Uncaught TypeError: undefined is not a function  

Everyone has seen this at one time or another. It isn't very helpful, and these types of errors are actually difficult to account for in JavaScript applications. The only error handling in JavaScript that the language allows is generic try/catch, where you can catch an error, but trying to recover from specific errors is difficult to do. Large in part the the fact that you can throw anything, not just errors. You can

Read More
filed under:  javascript oop errors inheritance

Summer of Sockets part 3: Pub Sub With ZeroMQ

2014 Sep13

S

o far we have looked at PUSH/PULL and REQ/REP. One of the more interested socket combinations in ZeroMQ is PUB/SUB. PUB/SUB has multitude of real world applications in distributed systems, Ranging from remote Work Queues, Push notifications for real time web applications, inter-application communications,
etc. Like other socket types found in ZeroMQ either each the pub and sub socket types can either bind or connect to an endpoint. It is really a matter of your applications use case. In general, the part of the application that will have the greatest up time would bind to a port and all others would connect.

There are two primary difference with the PUB/SUB sockets over the previously

Read More

Summer of Sockets Part 2: Request and Response With ZeroMQ

2013 Dec29

Z

eroMQ makes it incredibly easy to connect and communicate between individual applications in distinctive patterns. Last time we took a look at the simple PUSH / PULL socket types. The next types we'll take a look at are the Request / Reply ( REQ / REP ) sockets. You can download the source code for the following examples if you would like to follow along.

Request & Reply sockets are probably the most familiar and easiest to reason about for web developers. This most obvious use case with REQ and REQ sockets, is a web server. A client ( REQ ) makes a request to the server ( REP ) and it waits until a response comes back. Unlike the PUSH socket, the REQ socket will wait until

Read More

Going "Web Scale" with JavaScript Polymorphism

2013 Jul09

T

he recent trend has been to push more and more work on to the client side as browser become more and more capable. This means that complexity of your javascript can get out of hand very quickly. Being able to scale simple piece up to handle complex and even specialized tasks is an important part of sound javascript application architecture.

One way to accomplish this is through simple polymorphism, or as in many Object Orient Orientated languages, Generic Programming. The general pattern is to have a single light class or module that has a standard set of methods which does nothing more than call methods of the same name on any number of other objects ( classes, modules, etc. )

For

Read More
filed under:  javascript amd polymorphism oop

Fun With MooTool Part 3: Class Mutators

2013 Jun23

T

he Class system in Mootools a exceptionally powerful for the little amount of code that it is comprised of. A large part of that power comes from the Mutator system. A Mutator can be thought of a lot like a plug-in. However unlike most plug-ins that are executed at run time, Mootools Mutators are executed at the time a class is actually defined altering the end result of the class

A Mutator is nothing more than a JavaScript function that has access to the internals of the class it is attached to. These functions are stored in a Mutators namespace in the Mootools library. Each mutator manifests itself as a special key on your class and the value of

Read More

Goin' Full Require: Turtles All The Way Down

2012 Nov15

N

ow that we are comfortable the syntax for creating and loading modules we need to set up configure RequireJS to load anything and everything we want for our applications. Luckily the configuration API for RequireJS is rather straight forward and allows for a lot of flexibility. Configuring RequireJS is a simple as passing it an Object literal.

var SiteName = require({ });  

Pretty simple! The require function returns a configured instance of RequireJS so you can stash it in a variable and use it as the loader for a specific set of modules, or as it if were your applications primary name space. Now, there are 3 options in particular we are going to want to pay special attention to -

Read More
filed under:  javascript amd requirejs