Charting with WPF/C#

Visualization techniques have been evolving rapidly and with a growing trend towards ‘big data’ and ‘analytics’ there are so many technologies to choose from. If the development technology is not a constraint, D3 is arguably the best way to go, but if you are looking for some charting capabilities within a WPF/C# development environment your free choices are somewhat limited. Based on what I have tried out so far and read in various forums, here are some popular ones:

  1. MS Chart Controls – Pretty decent collection of controls, but you need to use ‘old’ Windows Forms to use these. You can host the Form within a WPF application if you really want.
  2. WPF Toolkit Charting Controls – I think this is the first real WPF charting control Microsoft released. Charting options are good, but they still look like the old Forms-based charts. Here is a good tutorial to get started.
  3. D3 1.0 – Pretty easy to get started, but I felt the number of options available was rather limited. First attempt to have D3 library capabilities in a WPF environment I think. There is a good tutorial here on how to get started with these charts. Seems like the WPF effort has somewhat been abandoned and there is more emphasis on having the D3 capabilities in Silverlight now which is called D3 2.0.
  4. Metro Charts – “This project provides a small library to display charts in Modern UI Style (formerly known as Metro) in WPF, Silverlight and Windows 8 applications”. This is the best I have seen so far and the UI looks much better and modern compared to all others. The trouble is, these are aimed towards Windows 8 applications and you need Visual Studio 2012 to run the sample code you get off the site. You still can get these to run on Windows 7 with Visual Studio 2010, and that’s what I’m going to focus on in this post.

- To get started, download the sample code from http://modernuicharts.codeplex.com/releases/view/106190 (if you don’t want to go through the steps yourself, just download the project

which contains all the changes I’m discussing here and you should be good to go! Just rename to a .zip – WordPress doesn’t allow me to upload zip files). You should have the following folders and files:

image

- Open the solution file. Permanently remove source control since we won’t be needing this.

image

- At this point you’ll get a bunch of errors starting from:

image

and your Solution Explorer will like this:

image

- Delete all projects except for the following three:

image

- Save the workspace and close Visual Studio

- Now, open Windows explorer and open the .csproj file in a text editor (like NotePad++) and change the ‘TargetFrameworkVersion’ from 4.5 to 4.0. You need to do this for the 3 files under De.TorstenMandelkow.MetroChart, De.TorstenMandelkow.MetroChart.WPF and TestApplicationWPF folders.

image

- If you want, you can clean up your folder a bit as well so that you have only the following on disk. This step is optional.

image

- Open the main solution (i.e., MetroChart.sln). Remove source control association bindings if a message pops up. You should see the following where the source code is now loaded into VS.

image

- Change startup project to TestApplicationWPF

image

- Hit F5, and you should see the following Smile

image

One of the things I wanted to do was find the underlying data when a user double clicks on a chart. To do this, you can add something like this in your xaml:

image

Then in your code behind, you can handle this event in whatever the way you want.

image

Play around. Using Metro Charts is fairly straight forward.

Posted in General | Tagged , , , , , , , , , | 1 Comment

Accessing NCBO Annotator Web Service in C#

The NCBO Annotator allows you to get annotations for (biomedical) text from a number of standard ontologies. For instance, if I want to find the corresponding RadLex codes for ‘abdomen knee’, I can type them into the textbox, and restrict the ontology list to just RadLex.

image

This manual approach works fine if you just want a few sentences annotated, but if you need to annotate multiple sentences in a systematic manner you need to do this programmatically. This is where the annotate web services come in. There are several client examples as well where you can find some sample code in several languages. There is a Java annotator client example, but unfortunately there is no C# example. For something I’m trying to implement I need to (or rather, I prefer to) use C#. Translating some Java code to C# isn’t hard, but in case someone’s looking for a ready-to-use C# example, following is a function/method you can use:

private void GetData()
        {
            Uri address = new Uri("http://rest.bioontology.org/obs/annotator");

            // Create the web request  
            HttpWebRequest request = WebRequest.Create(address) as HttpWebRequest;

            // Set type to POST  
            request.Method = "POST";
            request.ContentType = "application/x-www-form-urlencoded";
            request.UserAgent = "Annotator Client Example - Annotator";

            String text = "abdomen knee";

            StringBuilder data = new StringBuilder();

            // Configure the form parameters
            data.Append("longestOnly=" + "false&");
            data.Append("wholeWordOnly=" + "true&");
            data.Append("filterNumber=" + "true&");
            data.Append("withDefaultStopWords=" + "true&");
            data.Append("isTopWordsCaseSensitive=" + "false&");
            data.Append("mintermSize=" + "3&");
            data.Append("scored=" + "true&");
            data.Append("withSynonyms=" + "true&");
            data.Append("ontologiesToExpand=" + "&");
            data.Append("ontologiesToKeepInResult=" + "1057&"); // from http://bioportal.bioontology.org/ontologies/. Use comma sep. list of ont. ids
            data.Append("isVirtualOntologyId=" + "true&");
            data.Append("semanticTypes=" + "&");
            data.Append("levelMax=" + "0&");
            data.Append("mappingTypes=" + "&"); //null, Automatic, Manual 
            data.Append("textToAnnotate=" + text + "&");
            data.Append("format=" + "xml&"); //Options are 'text', 'xml', 'tabDelimited'   
            data.Append("apikey=" + "YOUR_KEY");

            // Create a byte array of the data we want to send  
            byte[] byteData = UTF8Encoding.UTF8.GetBytes(data.ToString());

            // Set the content length in the request headers  
            request.ContentLength = byteData.Length;

            // Write data  
            using (Stream postStream = request.GetRequestStream())
            {
                postStream.Write(byteData, 0, byteData.Length);
            }

            // Get response  
            using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
            {

                XmlDocument doc = new XmlDocument();
                doc.Load(response.GetResponseStream());

                XmlNodeList nodes = doc.SelectNodes("/success/data/annotatorResultBean/annotations/annotationBean");
                foreach (XmlNode node in nodes)
                {
                    string radlexId = node.SelectSingleNode("concept/localConceptId").InnerText.Split('/')[1];
                    string radlexDescription = node.SelectSingleNode("concept/preferredName").InnerText;
                }

                // Or print to console [uncomment line: doc.Load(response.GetResponseStream());]
                //StreamReader reader = new StreamReader(response.GetResponseStream());
                //Console.WriteLine(reader.ReadToEnd());
            }
        }

You can set breakpoints at radlexID/radlexDescription to see that what you get programmatically is exactly the same as what you get when you type the text directly into website. Per the comment in the code, pick the list of ontologies you are interested in from http://bioportal.bioontology.org/ontologies and pass it as a comma separated list (e.g., if I’m interested in RadLex and Snomed, I’ll use:
“data.Append(“ontologiesToKeepInResult=” + “1057,1353&”);
where 1353 is the ID for Snomed.

Before you can use the web service, you’ll need to sign up at http://bioportal.bioontology.org/accounts/new and get an API key first (which you will then pass as the value for ‘apikey’).

Posted in General | Tagged , , , , , | 1 Comment

Extracting RadLex codes and descriptions from ontology

I was trying to find a flat list of RadLex terms that just had the IDs and descriptions. You can easily browse RadLex using the RadLex Term Browser,  but my requirement was to do a quick lookup using a local database. I couldn’t find something online that was readily available, so I wrote some code to extract just the Id, description fields from RadLex.owl (which you can download from http://www.radlex.org/). In the attached file you can find the list of 34k+ RadLex terms. It’s an Excel file, so you can easily import it into a SQL Server or MySQL  database. I can’t upload a tab delimited .txt file to WordPress.com (see http://en.support.wordpress.com/accepted-filetypes/), so  if anyone needs the file in a different format, just let me know.


8-Nov-2012 update: Seems like there’s a new SQL script that contains all RadLex terms – ftp://ftp.ihe.net/RadLex/documentation.html. I haven’t used it yet, but might be better/newer/more complete than the original Excel file I put together.

Posted in General | Tagged , , | 5 Comments

Good to be Greened!

There’re many many forums that discuss various aspects related to the greencard process. One of the recurring questions seems to be how many publications you need for EB1-EA/NIW. My case is neither since it was filed under EB1-OR., but thought I would just share with you what I had at the time of applying in case someone finds it useful. My impression/understanding based on various forums I’ve read is that the criteria for EB1-EA/NIW is a lot harder since these categories don’t require a job offer. Anyway, this is what I had:

  1. PhD in Computer Science (completed in 2010)
  2. 16 journal publications (9 first author) – mostly international fairly high-impact journals.
  3. 12 conference papers (6 first author)
  4. 13 reviewer invitations (6 direct, 2 on behalf of supervisor, 5 rejected)
  5. 12 awards (8 national/university level awards related to PhD work, 3 travel grants, 1 international recognition where a paper was selected to be included in the Yearbook of Medical Informatics)
  6. 4 emails/threads showing where other researchers had reached out to me asking for my advice/opinion
  7. 103 Google Scholar hits
  8. 60+ citations (excluding self citations) from 17 different countries – got this from Scopus
  9. 14 Pubmed entries (good journals are indexed in Pubmed if you’re in the medical domain)
  10. 2 provisional patent applications (i.e., not granted yet)
  11. 4 recommendation letters (2 from PhD supervisors, 1 from internship mentor, 1 from my project leader in the company)

You need to meet only two out of the following requirements for EB1-OR:

  • Evidence of receipt of major prizes or awards for outstanding achievement
  • Evidence of membership in associations that require their members to demonstrate outstanding achievement
  • Evidence of published material in professional publications written by others about the alien’s work in the academic field – (citations?)
  • Evidence of participation, either on a panel or individually, as a judge of the work of others in the same or allied academic field – (peer-reviewed?)
  • Evidence of original scientific or scholarly research contributions in the field
  • Evidence of authorship of scholarly books or articles (in scholarly journals with international circulation) in the field – (publications?)

I’m not totally sure of the difference between the last two since you can’t have publications unless you have original research. So I think you can’t have the last on its own, but if you can show that you have original research that hasn’t been published (yet?), then the one before last could be a standalone. So my guess is that if you have publications, you have already met the last two. Anyway, in my case, I think I met all except for the association membership criterion, although I may have been somewhat thin on the first (since my awards are mainly local).

This was the timeline from filing to approval:

11 April 2012: Filed I-140 under Premium Processing

13th April: USCIS received application – 2 days

16th April: I-140 approved (what a surprise..approved in 3 days!) – 5 days

30th April: I-485 filed – 19 days

19th May: Received EAD-AP combo card in the mail – 38 days

20th June: Had biometrics done – 60 days

20th July: I-485 approved for me; RFE for wife – 90 days

26th July: Permanent resident card arrived in mail – 96 days

10th August: Response for RFE filed

15th August: RFE accepted. Card ordered.

20th August: Card received

The RFE was a bit silly since we had submitted these documents already. Turns out it’s pretty common for the spouse to get an RFE. This is what we had:

  1. Please submit a copy of your official birth certificate. If the document is not in English, please include a certified English translation. If a birth certificate does not exist or cannot be obtained, you must demonstrate this and submit secondary evidence pertinent to the facts at issue.
  2. Submit a copy of the biographical page of your passport.

The Fragomen lawyers handled the entire process and it all took roughly 3 months from I-140 filing to I-485 approval. I was pleasantly surprised by the whole process although gathering the documents and all was a pain!  The lawyers + their team was top notch and made the whole process very smooth. Once USCIS made a mistake in my country of citizenship and even before I pointed this out, they had proactively taken steps to address this. On another occasion, they took care of a translation that was required – when the translation was done, there had been a spelling error and they had already gone back to the translator to fix the error. No idea if they handle individual cases, or what the charges are, but if anyone’s looking for a good lawyer/law firm, I can highly recommend these guys.

Note: Post is backdated since I don’t want this to be the first post in the blog

Posted in Personal | Tagged , , , , , , | 1 Comment

Connecting to SQL Server from Java

Almost every time I try to connect to SQL Server from Java I end up hacking something together, like copy-pasting the connection string from an old project; then again, more often than not I end up getting some sort of error and spending quite sometime Googling. So I spent sometime trying to understand the different options and thought I’d put together a few things I have learnt. I’m using eclipse, so the examples are based on that, but I’m sure you could easily find the equivalent if you’re using a different editor.

There seem to be primarily two ways to connect to SQL Server – using Windows Authentication or SQL Authentication (with a username/password). Also, we have two commonly used drivers – JDBC and JTDS. This gives us four combinations and here are the connection strings/settings you need to get things up and running (you may need to adjust the string according to your PC name, SQL Server instance and DB name):

  1. Windows authentication + JDBC
This is the connection string we have to use:
String url ="jdbc:sqlserver://MYPC\\SQLEXPRESS;databaseName=MYDB;integratedSecurity=true";
Class.forName("com.microsoft.sqlserver.jdbc.SQLServerDriver");
Connection conn = DriverManager.getConnection(url);

For this to work, you’ll need to have sqljdbc driver included in the build path:
image
Note that you need to point to the sqljdbc4.jar (shown below) in ‘Microsoft JDBC Driver 4.0 for SQL Server’ folder. Make sure you have all other files that go along with it as well as simply having only the .jar won’t do.
image
You would think this would be enough, but if you just run it now you’re going to get the following error:

Jan 22, 2012 1:41:43 PM com.microsoft.sqlserver.jdbc.AuthenticationJNI <clinit>
WARNING: Failed to load the sqljdbc_auth.dll cause :- no sqljdbc_auth in java.library.path

So you also need to add the path to sqljdbc_auth.dll as a VM argument:
image
Now at one stage I was having the 3.0 JDBC driver and was getting the same error above and after wasting a decent amount of time found out that SQL Server 2012 requires the JDBC 4.0 driver. So if you are working with SQL Server 2008 you might need 3.0, although I’m guessing 4.0 should be backward compatible.

2.  SQL authentication + JDBC

This is probably the easiest scenario. You need to add sqljdbc4.jar similar to (1), but don’t need the VM arguments. The connection string to use in this case is:

String userName ="username";
String password ="password";

String url ="jdbc:sqlserver://MYPC\\SQLEXPRESS;databaseName=MYDB";

Class.forName("com.microsoft.sqlserver.jdbc.SQLServerDriver");
Connection conn = DriverManager.getConnection(url, userName, password);

Some details related to setting up a new user can be found here.

3.  Windows authentication + JTDS

JTDS uses Windows authentication by default, so this is the connection string we need to use (don’t need to explicitly mention ‘integrated security’).

String url ="jdbc:jtds:sqlserver://MYPC/MYDB;instance=SQLEXPRESS";
Class.forName("net.sourceforge.jtds.jdbc.Driver");
Connection conn = DriverManager.getConnection(url);

You also need jtds.jar in the Java build path:
image
Now at this point if you just run it you are going to get the following error:
java.sql.SQLException: I/O Error: SSO Failed: Native SSPI library not loaded. Check the java.library.path system property.
Took me a while to figure out how to resolve this, but what you need to do is something a bit similar to (1). Simply have the following as a VM argument:
image

’Path’ above is where your SSO folder is after installing the JTDS driver. Seems like for Windows authentication to work, Java needs to know where this ntlmaauth.dll is.

image

4.  SQL authentication + JTDS

Very similar to case (2). No VM arguments needed, but still need jtds.jar in the build path. Connection string to use is:

String userName ="username";
String password ="password";
String url ="jdbc:jtds:sqlserver://MYPC/MYDB;instance=SQLEXPRESS";
Class.forName("net.sourceforge.jtds.jdbc.Driver");
Connection conn = DriverManager.getConnection(url, userName, password);

Once you have the Connection, you can run a query/stored procedure as follows:
Query/UDF
try{           

    PreparedStatement ps = conn.prepareStatement("select * from yourTable/UDF/query");
    /* if UDF, and need to pass params, can do something like:
    ...prepareStatement("select * from UDF('" + UDFinputVal + "')" */

    ResultSet rs = ps.executeQuery();

    while (rs.next()) {

        String relatedConcept = rs.getString(1);
        int shortestDistance = rs.getInt(2);
        float weightedDistance = rs.getFloat(3);

        // do stuff    }  

    rs.close();
    ps.close();            
}
catch (Exception e) {
    e.printStackTrace();
}

Stored procedure
try{           

    // execute SP that requires two string arguments    PreparedStatement ps = conn.prepareStatement("exec storedProcedure ?,?");
    ps.setEscapeProcessing(true);

    ps.setString(1, "str1");
    ps.setString(2, "str2");

    ResultSet rs = ps.executeQuery();

    while (rs.next()) {
        String strVal = rs.getString(1);
        // do stuff    }

    /* If you get multiple results back from you SP, you can iterate through them using:

    ps.getMoreResults();
    rs = ps.getResultSet();

    while (rs.next()) {
        String strVal = rs.getString(1);
        ... 
    }  */

    rs.close();
    ps.close();            
}
catch (Exception e) {
    e.printStackTrace();
}

Keywords: com.microsoft.sqlserver.jdbc.AuthenticationJNI, connecting to SQL Server from Java, eclipse connect to SQL Server, Failed to load the sqljdbc_auth.dll cause :- no sqljdbc_auth in java.library.path, java.sql.SQLException: I/O Error: SSO Failed: Native SSPI library not loaded. Check the java.library.path system property, jdbc, jtds, SQL Server Windows authentication, executing SQL Server stored procedure from Java, calling SQL Server user-defined-function (UDF) from Java, SQL Server multiple tables ResultSets

Posted in General | Tagged , , , , , , , , , , , , | 81 Comments