A server application
may be started on one of several
server hardware platforms. Although
each platform/server must be assigned
a separate IP address, it is unreasonable
for client applications to be changed
each time the server platform changes.
Similarly, if the server hardware
box fails, it is desirable from
a disaster recovery perspective
that the server application be quickly
started on a standby server and
that the client applications not
Dynamic discovery includes client
and server discovery components
that are embedded in the client
and server applications.
- The server component is assigned
an instance name before being
- Optionally, it may be assigned
a specific port address on which
to listen for remote TCP connections.
- When the server's discovery
component is opened, it begins
listening on an IP multicast group
for a "where-is" request
that matches its instance name.
Whenever a corresponding "where-is"
request is detected, the discovery
server component responds by notifiying
the client of the host address
and port number that should be
used for a connection.
An application employs
a fifo to handle the loosely coupled
flow of data between two application
threads (A and B). Most of the time,
the data provider (thread A) is
processing a moderate amount of
information that can be processed
quickly by the data consumer (thread
There are peak periods during the
day when thread A generates data
that is orders of magnitude greater
than what can be handled by thread
B. If the information in the queue
were to remain memory resident,
the application could fail due to
a memory overflow.
Thread B is responsible for writing
information to a relational data
mart, and occassionally the DBMS
has failed or the DBMS has become
deadlocked. As a result, the application
was shut down. However, the data
still stored in the queue could
not be lost, and must be available
to be processed the next time the
application is started.
Paging fifos are muti-threaded
fifo queues. The queues support
reading and writing of string objects,
or POJO objects that can be converted
into strings. There can be multiple
writers, but only a single reader.
- When the fifo is opened, it
can be provided an instance name.
If provided such a name, the data
contained in the queue will be
persisted in compressed format
when the queue is closed, and
will available the next time that
named fifo is opened simply by
utilizing the same instance name.
- Additionally, each queue has
a memory capacity threshold that
is configurable for each fifo.
Until that threshold is reached,
all the fifo elements are stored
in memory. Once the threshold
is reached, entries are paged
(written) to disk. This feature
enables applications to read and
write to the fifo without the
overhead of disk IO, but memory
will not be exhausted in cases
when the fifo size reaches what
is considered a maximum memory
Peer to Peer
A hub and spoke application
is consolidating multple sources
of data. The hub application is
essentially a singleton that is
provided an instance name of "datahub."
The hub is responsible for writing
all of the information provided
by the various hubs to a common
data repository. The data repository
supports both adds and updates,
and as a result the hub requires
that what is written was received
in fifo order.
Peer to peer async
supports a loosely coupled interprocess
- The data consumer node (reader)
is a multi-threaded class that
is provided a unique instance
name and is discoverable by that
name. (See Dynamic Discovery).
- The reader supports simultaneous
attachments by multiple data provider
(writers), and all the data written
to the reader is cached in a single
queue (see paging fifo).
- The discovery component set
is implemented in IP multicasting,
and the data flow is implemented
in reliable TCP/IP.
- When the hub is executed, it
instantiates a data discovery
server that first tests that there
is no other application calling
- If the discovery server finds
that there is one, it throws an
exception and exits.
- If there is no other instance
with that name, the discovery
server begins to listen for writer
spokes that want to attach to
- When a spoke attempt to find
the datahub, the discovery server
replies to the spoke with the
ip address and server port of
- The spoke node then creates
a TCP socket session with to the
hub, and begins passing strings
to the hub as data packets.
- At any time, the spoke or hub
can terminate the session
An workflow application
is designed to contain a set of
loosely coupled components that
execute as separate units of work
The various components
are instantiated in a specific order
by a controller class that notifies
each thread when it must initialize
itself and then begin its own unique
iteratirve process. When the time
comes for the application to terminate,
each component is notified to perform
whatever cleanup is indicated, and
then to stop processing. Additionally,
if a component encounters a fatal
error situation, it is responsible
for terminating its processing and
notifying the application controller
of its termination.
Since there is more
than one stack within the application
(one for each thread), rather than
thowing exceptions, each component
uses a special multi-threaded mechanism
to communicate state information
and error conditions to the controller.
A server application
is executing on a remote host computer.
There is the need to monitor the
state of the application is real
time through some sort of graphics
monifor. There may be one or more
individuals that are monitoring
the sample application.
- A threaded component periodically
wakes up (typically every 10 seconds)
and interrogates the state of
the application, accessing data
elements from various classes,
formatting the information into
a delimited string, and then publishing
the information to an IP multicast
- Corresponding listeners that
subscribe to that multicast group
are passed the string, decode
the message, and pass the information
to a method that in turn writes
the data to labels and tables
on a graphic console.
In addition to logs
that identify warning and catastrophic
events, there is a need to generate
another set of readable logs that
are more than status oriented. Logs
might include entries for all rejected
records, all records in a pre-processed
state, or a chronology of significant
events. The data these logs contain
are useful for audit purposes, to
replicate an application run for
purposes of debugging, and for subsequent
regression testing releases of the
Logs should be readable
by a text editor. They may or may
not contain a date and time stamp.
- There are logs defined for logging
rejected records, safe storage
of data, application audit, generalized
errors, and for application-defined
- Logs can be directed either
to standard outputs (stderror,
stdout), or to files.
- The standard output log can
also be redirected to a scrollable
Copyright (c) 1999-2011 George White and
All rights reserved.