lasticsearch is an extremely powerful searchable document store. The more you use it, the more you learn about the more realize how deep the rabbit hole of possibility goes. Except for when it comes `null` values. The Achilles heel of elasticsearch. What it really boils down to is elasticsearch doesn't index `null` values or fields that are missing from a document. Even if you have set up an index mapping and told elasticsearch about your field.

As you might think it can get a little tricky searching document for a field value or where that field is null / missing - especially when combined with other field queries. For example lets say I have some documents that looks something like

Read More

lasticsearch is an amazing piece of technology. Built on top of luecine it offers all of he incredible search facilities that you'd expect from a full featured search. What makes elasticsearch so powerful, however, is the fact that it stores the actual data that was originally index as JSON documents. Basically, it is a Full Text Search Database more so than a search engine. This allows elastic search to do things that other search engines can't do like aggregations, scripted queries, multi-query searches, etc; All in addition to the expected searching capabilities like suggestions, spelling corrections, faceting, and so on. For these reasons people are using elasticsearch as the primary data store for massive amounts of data.

One thing

Read More

ately I have been using Docker for development. It makes it really easy to stand up services and package entire stacks for projects. Most recently, I set up an elastic search instance, and the first thing I wanted to do was install the insanely helpful Head and Big Desk plugins. The official elastic search Dockerfile is just elastic search, no plugins. And of course, containers are pretty much a black box. Every time it restart, everything is back to square one. Luckily, The image exposes a volume for persistent data, and that is all we really need to have to install plugins. Here is how You can do that.

Install Elasticsearch Container

First we need to pull the docker

Read More

one of the things that makes ghost great is it's almost forced simplicity. However, with that comes a bit of rigidity. If you want something more than the provided simplicity, you might find your self pulling your hair out. Search is one of the thing that is lacking from the default ghost set up. The last incarnation of my blog had everything indexed in a Xapian search index. I really wanted to bring back a local search index. Luckily, in the 0.5.x series, they pulled some string so that you can use ghost as a plain npm module. This means you can build your own app around ghost. This should make search

Xapian is a search library

Read More

t has been about 3 years since I launched my blog. At the time the tech was pretty interesting. It was a Django project on mongrel2 app server backed by ZMQ and xapian as the search engine. I built out a modular Web UI with a WYSIWYG editor using mootools. But it was rather clunky in how it actually generated HTML.

Out With The Old

The WYSIWYG Editor was probably the biggest problem. It took serious liberties in cleaning the HTML and usually resulted in me editing the code by hand in the end. Interesting code blocks with highlighting was a 3 step process and was almost always formatted wrong.

Managing Images was a real pain as well. Dealing

Read More

o consider myself to be an equal opportunity coder. I like investigating different tools built in different languages to solve different types of problems. Recently at work I had the opportunity to play around with elasticsearch. Elasticsearch brings the whole distributed / non relational data craze to the world of search. It is actually a rather impressive piece of tech. If you are at all interested in search as a problem, I recommend you give it a look. In any event, I need to get some data into the search engine to play around with. Where could I get a good deal of data without doing the heavy lifting myself? Twitter! Sure the twitter activity stream. I can rig that

Read More