mixed storage applications (redis and rdbms)

Gathering real time data and showing various statistics is a plus for each software product on the market. The same goes for our logistics projects we are doing. Here you have metrics like picked articles, number of orders, orderposition analysis and others. While some metrics are easy to gather and easy to visualize via normal RDBMS, others have quite a big impact on db read and write performance. Of course you can chose OLAP to visualize various statistics but this wouldnt be realtime anymore. We want to show stats as they are happening with the frontend techniques you have nowadays like Websockets or AJAX poll. If you do classic ajax polling for expensive stats (SQLs) in a normal db, you simply kill your system. Then you can start putting massive efforts into clustering rdbms or change the way you think about databases for certain features.

With redis, being an in-memory key-value datastore with persistence capabilities, you get insane performance. The only thing you need to be aware of is the projected size of the resulting datastore. You really dont want the OS swapping memory. But with some calculations and proper usage of time-to-live (TTL) of your entries, this can be handled.The good thing for real-time statistics data is, that its normally not business critical so if you start using redis in that area, you dont need to think about the persistence thing too much in the first run.

One weak point of redis is: Its quite new and you dont get a book on redis. redis itself is pretty simple from command set perspective but a best-practices book on building the best data structures for common use-case scenarios would be nice. Of course you can google a lot in that area (notably the blog of the redis author) and most solutions are not that hard to create yourself. In terms of API libraries you get at least two quite mature ones in java with jedis and jredis. Plus you can also get some kind of higher abstraction library with Spring-Data Redis which support both possible base libs.

NoSQL databases are eveywhere around and this opens a new world of possibilities. IMO they will never replace rdbms in database centric apps like erp or crm systems but they will help where rdbms are not performing that well. So in the future we will have more mixed-storage applications which will use the correct storage for the needed feature.

What we have done so far ;-)

In one of my last posts i mentioned all the things we wanna focus on technically. Lets see what we achieved so far:

IP Centrex for telephony (cloud based VoiP): After switching to Placetel with our company numbers, we were shocked about the quality of the "phone line". We had to redial several times to get through and we got hang ups right in the call. All this for serveral weeks. We decided to quit that provider and switched to the next IP Centrex provider called Teamfon. BTW, switching carriers is always a nightmare because of the number porting process. Now with Teamfon we are very happy with the service and the quality of the lines. DONE!

jQuery: We are building more and more systems with jQuery. In fact we dont use anything else currently for front ends. DONE!

Versioning with GIT: We moved all projects to GIT. Currently we are using Codebase as hosting and project management platform. Its not pefect but they really offer much for the money. DONE!

Reduce Soap, use REST: When we build new services, we really count on Rest nowadays. But we have tons of apps on customer site which still use Soap though. Lets call it MORE-OR-LESS-DONE!

Websockets: Nada, nothing. FAILED SO FAR

Android App: Dismissed in favor of HTML based phonegap apps. CHANGED_MIND