Sunday 26 May 2013

IsectMP

IsectMP is a middleware multiplexer written in ANSI C that uses blocking RPC's in a single threaded select loop. The goal of IsectMP is to solve the client/server concurrency problem using collections of identical processes rather than threads with the following constraints:

1.    Keep the API as simple as possible to promote language agnosity.
2.    Impose no record structure on the data being exchanged.
3.    Have all data transfer move through a central broker (Isectd).
4.    Be able to stop and start remote processes via Isectd.

1.    IsectMP only wants to solve the Request/Response model.  This is the classic client/server problem where you have a single static server and many clients that come and go.
2.    IsectMP deliberately uses blocking I/O on a single thread in Isectd.  It solves concurrency by breaking down responses from workers into a stream of discrete chunks which must be concatenated at their destination. Requests from clients can be chopped up too, but this is unusual when dealing with databases as requests are usually small while responses can be very large.
3.    IsectMP deliberately forces all communication to pass through Isectd.  While some may argue this is creating a bottleneck, the advantage of having a central broker is that administrators know the health of the entire application by querying Isectd's registers.
4.    Isectd comes supplied with a command language that lets you adjust its state dynamically.
5.    IsectMP comes supplied with a parent daemon (Execd) that can start child processes locally from an instruction sent remotely via Isectd.

It turns out there are lots of other open source libraries out there doing similar things, but they all have different goals in mind from IsectMP.  I freely admit to only knowing IsectMP well, but from what I can see, if you stick with open source:

1.    IsectMP is the only one that uses ANSI C and compiles on Windows without issue.
2.    IsectMP deliberately uses a central broker.  The others frequently try to avoid using a central broker.
3.    IsectMP has the smallest API of any, just five function calls.
4.    IsectMP comes supplied with its own console client that can configure and control the central broker remotely.
5.    IsectMP's two daemons are tiny, the largest daemon weighs in at < 250kB including DLL's.

Certainly IsectMP has it's failings:

1.    It is not asynchronous.
2.    It will only cope with hundreds of clients, since each client gets it's own dedicated socket.
3.    The only transport mechanism it can use is TCP.
4.    It is single threaded.
5.    It's API relies on a C struct which causes headaches when creating new language bindings (2021-08-22  This is no longer true. A lot of reworking to the internals of Isectd has been done since this post was written).

I would not attempt to use IsectMP for an enterprise size application, but for a department size application, IsectMP works very well indeed (2021-08-22 It also does a grand job acting as the middleware behind web sites).