Multi tiering for Financial Applications

Multi-Tier Application Architecture isn’t a new concept to anyone who has done any sort of enterprise development to the point nobody ask about this during technical interviews anymore.
At minimum, there’re always three basic tiers whether you’re building web application or client server application:
a. Presentation
b. Application – Business logic
c. Data source
For financial applications, where do you put your calculations? That’s a matter of debate (but it shouldn’t).
I can’t tell you how many times I have seen applications gets built using the standard cookie cutter: DAO to load records from database into entities in Application tier, fashionably complying to every standard practice using OR mapper such as hibernate, Repositories and DAO with Spring. I’m not sure if people do this to learn the different technologies? To comply with Golden OO design paradigm. Or too afraid to deviate from “Best Practice”. This pattern simply don’t apply to all scenario. Not just “edge cases”.
For starter,
a. What kind of calculation are you running? Derivatives risk and pricing? VAR? Stressing? Time series analysis, covariance calculations, factor construction in portfolio optimization? Theses are computationally intensive. Quant libraries generally in c++, Python,  Java. And typically load distributed and calculations, thus, done in “Application Tiers”.
Or are you running simple pnl updates, aggregate position level pnl/return/risk to book level? funding or trade allocation? Reconciliation? These are simple mathematics (no quant Library) : key concatenation/matching, simple aggregations. This brings us to next point.
b. Data volume, performance, and proximity to data source. If your calculations sources or operate on a lot of data, unless nature of calculation complex. Or that it requires quant libraries. There’s probably very little reason why these should be done in Application Tier. Databases are extremely good at keys concatenation /matching, aggregation and simple arithmetic. If data already in database, you’re processing more than a few thousand rows, performance gains can be realised by running these calculations in database/SQL. Even if you have multiple data sources (even message bus or non-SQL sources) : One can always build simple data feeds, consolidate into single database. Downside to this approach is, SQL not portable across different database vendors.
c. Support
If calculations done in SQL, this means production trouble shooting can be done without a debugger. What this further means is that Level One support don’t need bother developers. More importantly, fixes can be simple SQL patches – no need recompile and redeploy, which adds to the risk.
d. Simplicity, Agile, and Maintainability
Let’s keep things simple. You’re adding complexity if you are doing simple maths in application tier, everytime you add a bean, entity, dao.
Happy Coding!


Install Tomcat on Fedora VM for @msdev

This is continuation from Previous article on how to create a Fedora VM, with SFTP installed/configured ( This article is for @msdev who’re unfamiliar with Linux environment.

STEP 1. Download Java SDK (it includes JRE). Then install Java on linux box under path /usr/java/jdk and /usr/java/jre
You may download from your Windows development box, then follow these instructions ( to upload package to your Linux server box via SFTP.
@msdev, if you’re not familiar with Linux commands, the following commands may be handy.
cd /usr
mkdir java
cd /home/johndoe
mv jdk-8u25-linux-i586.gz /usr/java
tar -xvf jdk-8u25-linux-i586.gz
Also in case if you want to delete something:
rmdir ./SomeDirectory (SomeDirectory must be empty)
rm -rf ./SomeDirectory    (non-empty directory)
rm SomeFile (Delete a file)
Also to set environment variables for java Temporarily:
export PATH=

To set the env var permanently, add the same line to ~/.bashrc.

export PATH=/usr/java/jdk1.8.0_25/bin:$PATH

STEP 2. Download and install Apache Tomcat under path /usr/apache/tomcat
Download Tar.gz package from here:

To configure JAVA_HOME and CATALINA_HOME, place a in the the /usr/apache/tomcat/apache-tomcat-7.0.57/bin directory with


STEP 3. Start Tomcat

Navigate to /usr/apache/tomcat/apache-tomcat-7.0.57/bin

From your Windows box hosting the VM, you can access the default webpage hosted by Tomcat. Test from Browser:

8080 is Tomcat default port, which can be changed from server.xml in conf folder.

Next, we’ll discuss how to develop a simple Java-WS (Web Service), put it on Fedora VM (on VirtualBox), the consume it from a dotnet Console Application on Windows box.

Happy Coding!

Java-WS and dotnet Interop Example

This article will show how to create a simple Java Web Service hosted in Tomcat (running on Windows), and consume the Java-WS from dotnet.

STEP 1. Download Tomcat


STEP 2. Set Environment Variables from Computer (Right click) \ Properties \ Advanced system Settings:

JAVA_HOME                        C:\Program Files\Java\jdk1.7.0_45

CATALINA_HOME            C:\apache-tomcat-6.0.43

Open *new* command prompt (existing command prompt wouldn’t see the new additions), verify settings correct:

echo %JAVA_HOME%

                                echo %CATALINA_HOME%

STEP 3. Start Tomcat

From command prompt, navigate to C:\apache-tomcat-6.0.43\bin

Then, from command prompt type: startup.bat

STEP 4. Verify it’s running

From browser:



From command prompt:

netstat -a

(You should notice port 8080 is taken)


STEP 5. Now, from NetBeans IDE, New Project




STEP 6. Add new Web Service

Simply enter Web Service Name “HellowWorldWebService” and Package “com.helloworld”, keep everything else default:




STEP 7. Test

Build the project:


Then Deploy (To Tomcat),


And finally, “Test Web Service” (By default when you create the web service, it’d create a “Hello” method automatically)


From your browser:


You’d later from Visual Studio add Service Reference to http://localhost:8080/HelloJavaWs/HellowWorldWebService?wsdl


STEP 8. At this point, with Tomcat running (And NetBeans IDE closed), you can create a new Console Application project from Visual Studio, then add Service Reference to http://localhost:8080/HelloJavaWs/HellowWorldWebService?wsdl




Now, run on debugger:


So, you think we’re done? No… NetBeans has a bug which will haunt you when you try deploy to your production server!

How to deploy Tomcat and the Java Web Service on a Linux box? First you may want to try how to setup a Fedora VM with FTP:

Then, you’d need install Tomcat & Java on your Linux box:

Then the rest of the steps are trivial –

I recap the steps here from the above Stackoverflow post, thanks to Thanh Phong (Just in case if someone deletes it!)

1. create the following dir c:\java\src\ws

2. create thew following file c:\java\src\ws\

// c:\java\src\ws\
package ws;
import javax.jws.WebService;

public class Adder {
 public double add( double value1, double value2 ) {
  return value1 + value2;

3. standing at c:\java\src\ execute

c:\java\src> javac ws\

file c:\java\src\ws\Adder.class will be generated

4. create the following directory structure with the following files



@msdev, you can download the jar files here

5. copy compiled file

copy c:\java\src\ws\Adder.class c:\tomcat6\webapps\adder_ws\WEB-INF\classes\ws\Adder.class

6. c:\tomcat6\webapps\adder_ws\META-INF\context.xml

<?xml version="1.0" encoding="UTF-8"?>
<Context antiJARLocking="true" path="/adder_ws"/>

7. c:\tomcat6\webapps\adder_ws\WEB-INF\web.xml

<?xml version="1.0" encoding="UTF-8"?>
<web-app version="2.5" xmlns="" xmlns:xsi="" xsi:schemaLocation="">
<!-- not needed

8. Config WEB-INF\sun-jaxws.xml

file : c:\tomcat6\webapps\adder_ws\WEB-INF\sun-jaxws.xml

<?xml version="1.0" encoding="UTF-8"?>
<endpoints version="2.0" xmlns="">
  <endpoint implementation="ws.Adder" name="Adder" url-pattern="/add"/>

9. Copy libraries

files at c:\tomcat6\webapps\adder_ws\WEB-INF\lib

copy netbeans files from

[netbeans dir]\enterprise\modules\ext\metro\*.*


[netbeans dir]\ide\modules\ext\jaxb\activation.jar

10. restart apache

Shutdown : c:\tomcat6\bin\shutdown.bat

Startup : c:\tomcat6\bin\startup.bat

11. Test

Open a web browser and go to http://localhost:8080/adder_ws/add?wsdl you can also use a tool like soapui ( to test the web service

Happy Coding!

Setup Linux VM with SFTP, Guide for @msdev

This article will quickly shows, in particular for #msdev who’re not familiar with Linux, how to set up a 32 bit Fedora VM with VirtualBox, and setup a SFTP server, then connect from Filezilla client from Windows machine hosting the Fedora VM.

STEP 1. Create Linux VM on VirtualBox. First, download vdi/vmdk image of Linux operating system. Google for “vdi vmdk Ubuntu” for example. Make sure you download an x64 image with username/pwd specified! Similar for SuSE, Fedora …etc

Ubuntu x64                           (username: osboxes, pwd:


Ubuntu_12.04-64bit.7z                        (username: vuser, pwd: vuser)



Fedora x86        (username: root, pwd: Pwl!ghsnts.)

This version has Desktop pre-installed.

    Active user account(s) (username/password): root/reverse, fedora/reverse


Then follow these instructions.

Don’t forget to check “Enable PAE/NX”


And add a second Network Adapter if your first Adapter is configured in NAT mode.



“Please check the network settings of the Ubuntu VM. If it is configured in NAT mode (which is the default mode), then you will not be able to access your windows 7 host from your VM nor will you be able to access your VM from outside. If that is the case, you can add another virtual network adapter to the VM and configure the network mode as “Host-only Adapter”, so that it would get an IP address in the series of “”. Similarly, host will also acquire a virtual NIC and IP address Now you would be able to communicate between the VM and the host using the 192.168.56.x network.”

Please note that if you enabled this Host-only adapter, you wouldn’t be able to for example ping

You wouldn’t be able to:

yum repolist all
yum info kernel

Instead you’d get error:

Error: Cannot retrieve repository metadata (repomd.xml) for repository: fedora. Please verify its path and try again

(Anyway, you’d still need to fix a few things for yum to work. REF:

Also, if you’re downloading the vm to test mono, try download a supported distribution (Ubuntu, SuSE for example):


Also, Right-CTRL is default hotkey for VirtualBox to exit mouse capture.

*** The following steps for Fedora ***

STEP 2. Create user account:

groupadd sftp_users

useradd johndoe

passwd johndoe             (Then it will prompt you to enter the pwd for ‘johndoe’. For this exercise, pwd=SomePassw0rd)

usermod -G sftp_users johndoe

STEP 3. Then config and start SFTP server like this:

cd /etc/ssh

vi sshd_config

Press INSERT to enter text in vi.

Press ESC when done. Then “:wq!” to save+exit

To set /home as root folder, modify “ChrootDirectory” setting using vi, then hit “wq!”:

ChrootDirectory /home

Also, comment out and add a line like below

#Subsystem sftp /usr/libexec/openssh/sftp-server

Subsystem sftp internal-sft

Then restart the service:

systemctl restart sshd.service

And try connect locally:

sftp johndoe@localhost

Then, try connect to VM from your Windows workstation hosting the VM? From VM command prompt, type ifconfig (not ipconfig) to identify IP address of your VM (2nd Network Adapter).


Then from for example, Filezilla client:


Happy Coding!

Teasing the Machine: Test Driving text-processing Sentiment Analysis, it’s fun.

Test driving

Respond is how “” Artificial Intelligence responded to the input text.

It’s fun.

1. Be a team player

Respond: Neutral

Remarks: That’s very wrong, whenever somebody re-iterate the Team Speech it’s generally negative, political.

2. Fuck me

Respond: Negative

Remarks: This really depends on … if you are a guy (a Player), or girl.

3. Fantastic we can all go home

Respond: Positive

Remarks: That’s obviously sarcasm, should be rated *Very Negative* instead.

4. what were you thinking

Respond: Negative

Remarks: This sounds about right. If somebody say this to you it’s most likely a reproach.

5. Just do it

Respond: Negative

Remarks: This really depends on connotation and context.

6. I’m with you

Respond: Positive As expected.

7. Your sister is pretty

Respond: Positive

Remarks: Again, it really depends on the context and who’s saying this.

8. My God

Respond: Negative

Remarks: Could have been “My God, thank you Lord with my heart”.

9. Jesus Christ

Respond: Neutral

Remarks: What?

10. Love me

Respond: Positive

Remarks: Again, it really depends on the context.

TSQL – Utility to read csv and Excel files

If you see this error?
The 32-bit OLE DB provider “Microsoft.ACE.OLEDB.12.0″ cannot be loaded in-process on a 64-bit SQL Server.

To resolve this error you need to get the “Access Database Engine 2010″ to facilitate the transfer of data between existing Microsoft Office files such as “Microsoft Office Access 2010″ (*.mdb and *.accdb) files and “Microsoft Office Excel 2010″ (*.xls, *.xlsx, and *.xlsb) files to other data sources such as “Microsoft SQL Server”.
Get the Microsoft Access Database Engine 2010 Redistributable []
IF EXISTS (SELECT * FROM sys.objects WHERE type = ‘P’ AND name = ‘sp_ReadCSVFile’)

IF EXISTS (SELECT * FROM sys.objects WHERE type = ‘P’ AND name = ‘sp_ReadExcelFile’)

SELECT * FROM sysobjects WHERE id = object_id(N’fn_SplitString’)
AND xtype IN (N’FN’, N’IF’, N’TF’)
DROP FUNCTION fn_SplitString


Remember to put a file Schema.ini in same folder as csv file. Also ‘@TargetTable’ parameter should be csv file name (but without Extension)

exec sp_ReadCSVFile ‘C:\tmp’, ‘EmpData’

COL2=FirstName TEXT
COL3=LastName TEXT
COL4=MiddleName TEXT
COL5=EmailAddr TEXT
@TargetFolder varchar(255),
@TargetTable varchar(255)
declare @SqlStmt nvarchar(max)

set @SqlStmt = ‘IF EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N”[dbo].[‘ + @TargetTable + ‘]”) AND type in (N”U”))’
set @SqlStmt = @SqlStmt + ‘ TRUNCATE TABLE ‘ + @TargetTable
PRINT @SqlStmt
exec sp_executesql @SqlStmt

set @SqlStmt = ‘ SELECT * INTO ‘ + @TargetTable
+ ‘ FROM OpenDataSource (”Microsoft.ACE.OLEDB.12.0”, ”Data Source=”‘ + @TargetFolder + ‘”;Extended properties=Text”)…[‘
+ @TargetTable + ‘#csv]’
PRINT @SqlStmt
exec sp_executesql @SqlStmt

exec sp_ReadExcelFile ‘C:\tmp’, ‘EmpData’, ‘EmpData.xls’, ‘[Sheet1$]’, ‘Col1,Col2,Col3,Col4,Col5’
@TargetFolder varchar(255), /* For example, ‘C:\tmp’ */
@TargetTable varchar(255), /* For example, ‘EmpData’ */
@ExcelFile varchar(255), /* For example, ‘EmpData.xls’ */
@ExcelSheet varchar(255), /* For example, ‘[Sheet1$]’ */
@ExcelFields varchar(8000) /* Comma separate list, for example: ‘Col1,Col2,Col3,Col4,Col5’ */
declare @SqlStmt nvarchar(max)
declare @FirstColumn nvarchar(255)

set @SqlStmt = ‘IF EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N”[dbo].[‘ + @TargetTable + ‘]”) AND type in (N”U”))’
set @SqlStmt = @SqlStmt + ‘ DROP TABLE ‘ + @TargetTable
PRINT @SqlStmt
exec sp_executesql @SqlStmt

select top 1 @FirstColumn=[part] from fn_SplitString(@ExcelFields, ‘,’)
print ‘FirstColumn: ‘ + @FirstColumn

set @SqlStmt = ‘SELECT ‘ + @ExcelFields + ‘ INTO ‘ + @TargetTable + ‘ FROM OPENROWSET(”Microsoft.ACE.OLEDB.12.0”,
”Excel 12.0;DATABASE=’ + @TargetFolder + ‘\’ + @ExcelFile + ‘;IMEX=1”, ”Select * from ‘ + @ExcelSheet + ”’)’
+ ‘WHERE NOT ‘ + @FirstColumn + ‘ IS NULL’
PRINT @SqlStmt
exec sp_executesql @SqlStmt

Taken from:
Usage: select top 1 part from fn_SplitString(‘aaa,bbb,ccc,ddd,eee’,’,’)
CREATE FUNCTION [dbo].[fn_SplitString]
@sString nvarchar(2048),
@cDelimiter nchar(1)
RETURNS @tParts TABLE ( part nvarchar(2048) )
if @sString is null return
declare @iStart int,
@iPos int
if substring( @sString, 1, 1 ) = @cDelimiter
set @iStart = 2
insert into @tParts
values( null )
set @iStart = 1
while 1=1
set @iPos = charindex( @cDelimiter, @sString, @iStart )
if @iPos = 0
set @iPos = len( @sString )+1
if @iPos – @iStart > 0
insert into @tParts
values ( substring( @sString, @iStart, @iPos-@iStart ))
insert into @tParts
values( null )
set @iStart = @iPos+1
if @iStart > len( @sString )

DataTable to-from HTML Table

Here’s a small handy utility which converts a C# DataTable to/from HTML Table. Happy Coding!

using System;
using System.Collections.Generic;
using System.Data;
using System.Globalization;
using System.Text;
using System.Xml;

namespace ClientUtil
public class DataTableUtil

public static string DataTableToXmlString(DataTable dtData)
if (dtData == null || dtData.Columns.Count == 0)
return (string) null;
DataColumn[] primaryKey = dtData.PrimaryKey;
StringBuilder stringBuilder = new StringBuilder();
foreach (DataColumn dataColumn in (InternalDataCollectionBase) dtData.Columns)
if (DataTableUtil.IsPrimaryKey(dataColumn.ColumnName, primaryKey))
stringBuilder.Append(“<TH IsPK=’true’ ColType='”).Append(Convert.ToString(dataColumn.DataType == typeof (object) ? (object) typeof (string) : (object) dataColumn.DataType)).Append(“‘>”).Append(dataColumn.ColumnName.Replace(“&”, “”)).Append(“</TH>”);
stringBuilder.Append(“<TH IsPK=’false’ ColType='”).Append(Convert.ToString(dataColumn.DataType == typeof (object) ? (object) typeof (string) : (object) dataColumn.DataType)).Append(“‘>”).Append(dataColumn.ColumnName.Replace(“&”, “”)).Append(“</TH>”);
int num1 = 0;
foreach (DataRow dataRow in (InternalDataCollectionBase) dtData.Rows)
int num2 = 0;
foreach (DataColumn dataColumn in (InternalDataCollectionBase) dtData.Columns)
string str = Convert.IsDBNull(dataRow[dataColumn.ColumnName]) ? (string) null : Convert.ToString(dataRow[dataColumn.ColumnName]).Replace(“<“, “&lt;”).Replace(“>”, “&gt;”).Replace(“\””, “&quot;”).Replace(“‘”, “&apos;”).Replace(“&”, “&amp;”);
if (!string.IsNullOrEmpty(str))
return ((object) stringBuilder).ToString();

protected static bool IsPrimaryKey(string ColumnName, DataColumn[] PKs)
if (PKs == null || string.IsNullOrEmpty(ColumnName))
return false;
foreach (DataColumn dataColumn in PKs)
if (dataColumn.ColumnName.ToLower().Trim() == ColumnName.ToLower().Trim())
return true;
return false;

public static DataTable XmlStringToDataTable(string XmlData)
DataTable dataTable = (DataTable) null;
IList<DataColumn> list = (IList<DataColumn>) new List<DataColumn>();
if (string.IsNullOrEmpty(XmlData))
return (DataTable) null;
XmlDocument xmlDocument1 = new XmlDocument();
xmlDocument1.PreserveWhitespace = true;
XmlDocument xmlDocument2 = xmlDocument1;
XmlNode xmlNode1 = xmlDocument2.SelectSingleNode(“/TABLE”);
if (xmlNode1 != null)
dataTable = new DataTable();
int num = 0;
foreach (XmlNode xmlNode2 in xmlNode1.SelectNodes(“TR”))
if (num == 0)
foreach (XmlNode xmlNode3 in xmlNode2.SelectNodes(“TH”))
bool result = false;
string str = xmlNode3.Attributes[“IsPK”].Value;
if (!string.IsNullOrEmpty(str))
if (!bool.TryParse(str, out result))
result = false;
result = false;
Type type = Type.GetType(xmlNode3.Attributes[“ColType”].Value);
DataColumn column = new DataColumn(xmlNode3.InnerText, type);
if (result)
if (!dataTable.Columns.Contains(column.ColumnName))
if (list.Count > 0)
DataColumn[] dataColumnArray = new DataColumn[list.Count];
for (int index = 0; index < list.Count; ++index)
dataColumnArray[index] = list[index];
dataTable.PrimaryKey = dataColumnArray;
DataRow row = dataTable.NewRow();
int index = 0;
foreach (XmlNode xmlNode3 in xmlNode2.SelectNodes(“TD”))
Type dataType = dataTable.Columns[index].DataType;
string s = xmlNode3.InnerText;
if (!string.IsNullOrEmpty(s))
s = s.Replace(“&lt;”, “<“);
s = s.Replace(“&gt;”, “>”);
s = s.Replace(“&quot;”, “\””);
s = s.Replace(“&apos;”, “‘”);
s = s.Replace(“&amp;”, “&”);
row[index] = Convert.ChangeType((object) s, dataType);
if (dataType == typeof (DateTime))
row[index] = (object) DateTime.ParseExact(s, “yyyyMMdd”, (IFormatProvider) CultureInfo.InvariantCulture);
row[index] = Convert.DBNull;
return dataTable;

That’s it. You may also want to check out how to convert DataTable to JSON string?

Taken’s from Mohamed Kamal Pharm’s article – first you convert DataTable.Rows into a List, the use System.Web.Script.Serialization.JavaScriptSerializer to render the list to a string:

public String ConvertDataTableTojSonString(DataTable dataTable)
System.Web.Script.Serialization.JavaScriptSerializer serializer =
new System.Web.Script.Serialization.JavaScriptSerializer();

List<Dictionary<String, Object>> tableRows = new List<Dictionary<String, Object>>();

Dictionary<String, Object> row;

foreach (DataRow dr in dataTable.Rows)
row = new Dictionary<String, Object>();
foreach (DataColumn col in dataTable.Columns)
row.Add(col.ColumnName, dr[col]);
return serializer.Serialize(tableRows);

You may also want to checkout how to serialize a graph to/from xml!