A Groovy parser for CSV files

Parsing CSV these days is pretty straight-forward and not a big deal especially when we have the handy libraries from Apache Commons (I’m talking bout Java world). In this post I will give you an example how to use the Apache Commons CSV with the magic of Groovy and its closures so it can look and feel a little more fun because parsing in general is job for sad people (not kidding).

We’ll make ourself a simple Groovy class that will hold a reference to the CVSParser file, and a reference to the headers and the current record/line of the file that we will iterate with the closure delegate set to the instance of this CSVParserUtils class.

Something like this:

class CSVParseUtils {

    CSVParser csvFile
    def record
    def headers

    CSVParseUtils(String fileLocation) {
        def reader = Paths.get(fileLocation).newReader()
        CSVFormat format = CSVFormat.DEFAULT.withHeader().withDelimiter(delimiter)
        csvFile = new CSVParser(reader, format)
        def header = csvFile.headerMap.keySet().first()
        headers = header.split(delimiter as String)
    }

As we can see it’s a constructor that takes the location to the CSV file that we want to parse, creates some default parsing format and generates new CSVFile that holds the CSV data.

As we see parsing is easy, but it’s better when we can transform the data on the run as we loop it. For that reason we will define a method called eachLine that will take a params Map and a Closure that will have access to the record/line instance and will do something with it.

/**
 * List each line of the csv and execute closure
 * @param params
 * @param closure
 */
def eachLine(Map params = [:], Closure closure) {
    def max = params.max ?: maxLines
    int linesRead = 0
    def rowIterator = csvFile.iterator()
    closure.setDelegate(this)

    while (rowIterator.hasNext() && linesRead++ < max) {
        record = rowIterator.next()
        closure.call(record)
    }
}

It’s nothing special only a simple loop that iterates through the iterator and calls the closure with the given record for that line as a closure argument.

How to use it?

def parser = new CSVParseUtils(fileLocation)
def result = [:]
// first 2 lines without header
parser.eachLine([max: 2]) { CSVRecord record ->
    result.put(record.recordNumber, record.values.size() > 4 ? 
             record.values[0..4] : record.values[0..record.values.size()])
}

We imagine that we need only the first 2 lines and the first 5 columns or something like that.

As you can see this closure loop is not specially connected with CSV, it’s just a clean way to iterate through any textual file line by line and do something with it. As a matter of fact you can use the BufferedReader which has method eachLine too.

The source code for this whole example can be found on github.

Thanks for reading.

Grails and SAPUI5 are friends

Hello reader,

instead of the planned walk through the city park and drinking some beer(s) mother nature swinging moods changed my plans and in place of the shiny sun gave me hard rain and sour mouth. In a situation like that, alone and bored I decided to bore you too and share this short text about two good friends called Grails and SAPUI5 (respect to the OpenUI5 project too). ūüôā

I’ve been working hard with the Grails framework this couple of years and different situations led me to different scenarios. Lately I found myself in situation that asked bringing closer the powerful SAP services to the web/mobile clients. And what is better that using the outsourced JavaScript MVC framework made by SAP called SAPUI5 or if you prefer the open source project name OpenUI5 in conjunction with the versatile Grails Framework.

If you’re familiar with Grails then you certainly know that with the latest 3++ versions of Grails there is great support for already established and pretty much famous frameworks/libraries AngularJS and ReactJS in forms of Grails app profiles and plugins. But there is no “official” support for interbridging SAPUI5 and Grails and that is the main motive for writing this blog post and sharing it with you.

SAPUI5 is a single paged application where all the magic is done with JS so what we need is a single html file or in this case a single gsp file. We use that file to define the paths to the SAPUI5 runtime (or sdk) resources and to init the main SAPUI5 application via short JavaScript code. SAPUI5 is best when used with the OData services and that its where this software shines, however it has also great support when working with JSON and provides us with swift JSONModel that we can use to fill up the application data. And because we have JSON then we must have the Restful Grails controllers that will provide us with well defined JSON.

So the situation is pretty simple: Grails connects us with the backend via web services (or else?) or it provides us the data on its own via GORM or something else. Then Grails transforms the data into a JSON format that is a sweet cake for the SAPUI5 to consume and make it look great both on web browsers and on any mobile clients (Smart phones, tablets etc.).

Well this won’t be worth a penny without a working example, right? Because that’s the cause I’ve published a little demo of Grails and SAPUI5 playing together that you can check it on github. In short words we have a Spring Security Core plugin for the authentication and authorization, the JSON Views plugin for making the JSON even easier and also an example how to make it work via rest based http calls if your clients is native app . And of course the SAPUI5 application itself.

Here’s the link to the repo.

Thanks for reading,

cheers.

 

 

Passing by ‘reference’ in Java

One of the first things that every Java programmer learns (or should learn) is that method parameters in Java are passed-by-value. That is the only truth and there is no so called ‘reference’ passing in Java. Every method call with parameters means that their value is copied in some memory chunk and then they are passed (the copied memory) to the local function to be used.

What is more important though is of what type is the parameter that is passed. Generally there are two different data types: primitive (int, char, double etc…) and complex aka objects (Object, array etc…). The thing that matters is what is the ‘value’ of them when they are used in parameter passing.

When we are passing parameters of primitive type we are passing the actual value of it. So if we pass an integer with value of 4, then the function will receive an integer with value of 4 as parameter value. However, if we pass parameter of complex type let’s say some object of class Company then the function will actually receive the pointer to real object location in memory. Or in Java terms, it will receive the copied value of the reference (address) to the Java object that we want to pass and use.

If in C++ we have: Company *c; to get the pointer , then in Java we have Company c; . It’s pretty much the same, the difference is how things are designed and implemented under the cover.

If we understand this, then we realize that even if there are no out parameters in Java when defining and implementing methods, we can still use the reference advantage to program that thing by our self.

To get things clear and imagine the picture we should actually see the picture, I mean the code.

For example we can use an array of size one to be our data holder. Passing and mutating data this way will change the real value that we want to be changed. An example code, try it:


package com.groggystuff;

/**
*
* @author Igor
*/
public class JavaArguments {

/**
*
* @param argument
*/
public static void mutate(int argument){
argument++;
}

/**
*
* @param argument
*/
public static void mutate(int[] argument){
argument[0]++;
}

/**
* @param args the command line arguments
*/
public static void main(String[] args) {
// example pass by value
int i = 4;
System.out.println("Value of 'i' before mutation: "+i); //prints 4
mutate(i);
System.out.println("Value of 'i' after mutation: "+i); // prints 4

// example pass by value too
int[] j = new int[1];
j[0] = 4;
System.out.println("Value of 'j' before mutation: "+j[0]); // prints 4
mutate(j);
System.out.println("Value of 'j' after mutation: "+j[0]); // prints 5

}

}

Thanks for reading,

I hope this post will be helpful to you in solving the coding mysteries in life, or something similar.

Please comment if you feel that  your comment is needed. Or comment at will, just to say hi for example.

Custom Authentication Success Handler with Grails and Spring Security

It’s Sunday and instead of devoting this day to our Lord I will dedicate it to the great Machine and its coding brethren. The jokes aside, this is a quick show up of how to establish custom Authentication Success Handler if you are working with Grails Framework + Spring Security Core Plugin.

Well first, why would you need to alter the ‘normal’ behaviour¬† of the handler?

The answer: let’s say, you want to change the targetUrl for the specific authenticated user. Is that not enough? ūüôā

HOW TO DO IT

With Spring and Java what you need to do is to implement the AuthenticationSuccessHandler interface. It has only one method to be implemented:

void onAuthenticationSuccess(HttpServletRequest var1, HttpServletResponse var2, Authentication var3);

With Grails and Spring Security Core plugin we follow the same path, just the ritual is a little bit different.

Spring Security Plugin use the AjaxAwareAuthenticationSuccessHandler¬† bean that extends SavedRequestAwareAuthenticationSuccessHandler and if you follow the hierarchy tree you will notice that at some point the needed interface is implemented at the upper classes. So what we need to do is just extend AjaxAwareAuthenticationSuccessHandler and define the bean in resources.groovy (–ĺr .xml).

package com.wordpress.groggystuff.grails

import grails.plugin.springsecurity.web.authentication.AjaxAwareAuthenticationSuccessHandler
import org.springframework.security.core.Authentication

import javax.servlet.ServletException
import javax.servlet.http.HttpServletRequest
import javax.servlet.http.HttpServletResponse
import javax.servlet.http.HttpSession

class GroggySuccessHandler extends AjaxAwareAuthenticationSuccessHandler {

    boolean userIsBadPerson = false

    @Override
    protected String determineTargetUrl(HttpServletRequest request,HttpServletResponse response) {

        if(userIsBadPerson){
            logger.info(&quot;This user is very nasty. Send him to /dev/null to rot.&quot;)
            return &quot;/dev/null&quot;
        }
        else {
            return super.determineTargetUrl(request, response)
        }
    }

    @Override
    public void onAuthenticationSuccess(final HttpServletRequest request, final HttpServletResponse response,
                                        final Authentication authentication) throws ServletException, IOException {
        try {
            checkIfTheUserIsBadPerson(request.getSession(),authentication)
            handle(request,response,authentication)
            super.clearAuthenticationAttributes(request)
        }
        finally {
            // always remove the saved request
            requestCache.removeRequest(request, response)
        }
    }

    protected void handle(HttpServletRequest request, HttpServletResponse response, Authentication authentication)
            throws IOException, ServletException {
        String targetUrl = determineTargetUrl(request, response)

        if (response.isCommitted()) {
            logger.debug(&quot;Response has already been committed. Unable to redirect to &quot; + targetUrl)
            return
        }

        redirectStrategy.sendRedirect(request, response, targetUrl)
    }


    private void checkIfTheUserIsBadPerson(HttpSession session, Authentication authentication){

        // do the groggy check to find if the user is a bad person
        // presume that the user is always a bad person
        userIsBadPerson = true
    }
}

When the user authenticates successfully onAuthenticationSuccess method is called. With this code the method determineTargetUrl will be always referenced when the user logins and from there we can easily change the targetUrl that the handle method redirects to. I wrote a logical check with which I check when to redirect and how to build my targetUrl in determineTargetUrl. Don’t forget to define the bean in resources.groovy .¬† The bean id must be the same as in the plugin, otherwise this class will be just ignored.

beans = {
    // other beans
    authenticationSuccessHandler(GroggySuccessHandler) {
        /* Reusing the security configuration */
        def conf = SpringSecurityUtils.securityConfig
        /* Configuring the bean */
        requestCache = ref('requestCache')
        redirectStrategy = ref('redirectStrategy')
        defaultTargetUrl = conf.successHandler.defaultTargetUrl
        alwaysUseDefaultTargetUrl = conf.successHandler.alwaysUseDefault
        targetUrlParameter = conf.successHandler.targetUrlParameter
        ajaxSuccessUrl = conf.successHandler.ajaxSuccessUrl
        useReferer = conf.successHandler.useReferer
    }
}

And that’s it my lads and gals. (tested with Grails 2.5.0 and spring-security-core:2.0-RC4)

And a song as always.

Cool usage of TimeCategory in Groovy

Groovy, the programming language based on JVM implements a feature called Categories. It is originally borrowed from Objective-C . Simple explanation for this feature can be the ability to implement new methods in existing classes without modifying their original code which in some way is injecting new methods through a Category class. For more information official documentation can be found here .

Rather interesting for me was playing with the TimeCategory class for writing a short and easy script for fixing some datetime columns in database. This class offers a convenient way of Date and Time manipulation.

General syntax for categories is the following:

use ( Class ) {
// Some code
}

Concrete usage of TimeCategory:

use ( TimeCategory ) {
// application on numbers:
println 1.minute.from.now
println 10.hours.ago
// application on dates
def someDate = new Date()
println someDate - 3.months
}

Seems weird? From when Integer has months, minutes, hours etc. methods ? Well it still doesn’t have any of that, however those methods are dynamically added with the TimeCategory use.

If you are interested how is this possible I suggest you to go through TimeCategory API and source code if possible. Also this forum post can be useful for deeper understanding of the groovy magic.

And last but not least, an example groovy script for your pleasure.


@GrabConfig(systemClassLoader=true)
@Grab(group='mysql', module='mysql-connector-java', version='5.1.27')

import groovy.time.TimeCategory
import java.sql.Timestamp

sql = groovy.sql.Sql.newInstance(
"jdbc:mysql://hostname:3306/DB_name?autoReconnect=true",
"user",
"password",
"com.mysql.jdbc.Driver")

def rows= [:]

// Select Data
sql.eachRow("select * from Table_Name"){
def impDates = new ImportedDates() // This is some custom Class found in the same package/directory if script
impDates.dateColumn = it.dateColumn

if(impDates.dateColumn!=null){
use(TimeCategory){
impDates.dateColumn = impDates.dateColumn - 1.day // Shift dateColumn for one day backwards in time
}
}

rows.put(it.UID,impDates) // Put private key and ImportedDate object in Map

}

// Update Data
rows.each {row-&gt;
ImportedDates id = row.value
// Check if value is different from null, if it is convert it to Timestamp(we use datetime column in db) and execute update query
dateColumn  = null
if(id.dateColumn) dateColumn = new Timestamp(id.dateColumn.getTime())

// Actual update query
sql.executeUpdate('update Table_Name set dateColumn = ? ' +
'where UID like ?',
[dateColumn, row.key.toString()])

}

Cheers.

Creating Java keystore from existing private key and certificate

With the bunch of programming codes and programs found on the Web this days, Code Signing Certificates are fact and necessity. But the people who are end-users or developers are still in process of adjusting the awareness what programs/scripts/code are safe and junk free and should be trusted before running them on local machine. With this little guide I want to help new people which are diving into this area of problems.

Different platforms offer different way for code signing their apps , and in this post I will focus just on Java based systems.

Java web and desktop apps are bound with keystore files that keep the certificate chains signed by Internet authorities. With this technique it is easy to make distinction from trusted to untrusted programs with investment of some time and money.

Generally the process of creating Java keystore that can sign applications(source codes) can be covered in couple of steps that include the client and the certificate issuer:

  1. The client creates keystore file and generates private and public key pair
  2. The client exports Code Signing Request from the keys with personal and trustworthy data
  3. The client sends the CSR to Certificate issuer and waits for approval. Normally it is contacted during pending time.
  4. Certificate Issuer sends to the client the signed certificate and probably additional intermediate/root chain certificates that need to be included into the keystore.
  5. The client imports the certificate (probably in pkcs7 format) into the original keystore that was used to generate the keys and CSR with the appropriate alias that was used during the creation of the keystore.
  6. The keystore is included in Java applications and referenced with the alias so to sign the JARs used in the apps.

However it can happen the client to receive private key that ought to be used, without previously creating a valid keystore and generating key pair within it. This received key was used for generating CSR and certificate request was already sent to authority.

Well at this point it gets confusing what is the next step that should be taken and is it possible this key to be used for creating a new keystore? Some will say it is not possible (and seems logical because keytool doesn’t allow it), you will need to create new keystore and generate key pair¬† and issue a new certificate request with the CSR exported from this keystore key pair. That’s not true though, there is always a way.

Let’s suppose the original request has been approved and you received valid certificate cert.crt. At this point you have private.key¬†and trusted¬†cert.crt.

This files need to be merged and exported into pkcs12 format with the help of libssl library.

openssl pkcs12 -export -in cert.crt -inkey private.key -certfile cert.crt -name <certificate(alias)_name> -out keystore.p12

Next this new generated keystore.p12 should be used to create new keystore in JKS format with the help of keytool from the JDK.

keytool -importkeystore -srckeystore keystore.p12 -srcstoretype pkcs12 -destkeystore keystore.jks -deststoretype JKS
And that’s it voila! We have created keystore in jks format from existing private key.

After we have the keystore needed it is easy to import new certificates if required. Example:

keytool -import -trustcacerts -alias <alias(certificate)_name> -keystore keystore.jks -file <certificate_filename>

That’s it, three commands that will make your life easier.

SkeletorLead

Cheers,

Igor

Setting Up Context in Apache Tomcat for Serving Static Files

The intro:

So I’ve heard you want to serve static files from your Tomcat Web App in a way that they won’t be deleted on WAR redeploy or Tomcat restarted?

You have a solution, and that is mapping a custom Context in your Apache Tomcat server.xml .

The scenario :

You have a site that allows users to upload images that are public,shared and not under a hood of some security filter. The most intuitive solution is to put them in some directory i.e. ‘uploads’ , but then you realize that the things in the exploded WAR rewrite on redeploy or if the war is in the webapps directory on Tomcat restart. (you can change this behaviour)

The solution is simple: save the files to some directory outside of the war (something like ‘/usr/share/tomcat/uploads’) and map that directory on the server context of your Tomcat AS (something like http://lesite:8080/uploads).

With workaround like this you will see your uploaded cute kitty picture like this: http://lesite:8080/uploads/kitty.jpg

The implementation:

Let’s use the same examples. The mapping is done in <CATALINA_HOME>/conf/server.xml (hopes you know what and where catalina_home is )

This is default situation on new Tomcat install (a snippet from sever.xml):

<Host name=”localhost” appBase=”webapps” unpackWARs=”true” autoDeploy=”true”>

<Valve className=”org.apache.catalina.valves.AccessLogValve” directory=”logs”
prefix=”localhost_access_log.” suffix=”.txt”
pattern=”%h %l %u %t &quot;%r&quot; %s %b” />

</Host>

But we want to change that in this:

<Host name=”localhost” appBase=”webapps” unpackWARs=”true” autoDeploy=”true”>

<Context docBase="/usr/share/tomcat/uploads" path="/uploads" />

<Valve className=”org.apache.catalina.valves.AccessLogValve” directory=”logs”
prefix=”localhost_access_log.” suffix=”.txt”
pattern=”%h %l %u %t &quot;%r&quot; %s %b” />

</Host>

And that’s it, end of setting. Restart , code and redeploy.

The cookie:

Java snippet of simple utilization:

public class UploadsServlet extends HttpServlet {

    @Override
    protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletExcpetion, IOException {
        File file = new File("/usr/share/tomcat/uploads", request.getPathInfo());
        response.setHeader("Content-Type", Files.probeContentType(file.toPath()));
        response.setHeader("Content-Length", String.valueOf(file.length()));
        Files.copy(file.toPath(), response.getOutputStream());
    }

}

The conclusion:
In exact things the need for conclusion is deprecated. Everything should be concluded the one way :

return goToTopAndReadAgain();

The hint:

Maybe you won’t be impressed, and probably you have a better solution/implementation for/of the scenario. However let me give you a clue how this can be found useful in different situation. Proxying and load balancing, possibly with Nginx on front and couple of Tomcats behind. Defining new server contextses and getting a feel of that damn Superman speed.

Fiuuuuuuuuuuuuuuuuu…

(salutations and thanks to a friend of mine for collaboration)

Deploying Java Applications: Apache Tomcat + IIS 7.5

I’m on vacation and I’m writing my first technical post. That’s unfortunate ūüôā

So this will be for all you lads and gentleman (and ladies) struggling in the world of Java and its ecosystem. Have you ever deployed Java apps on IIS? Well If I am the man to be consulted I’ll never choose that solution. But of course here comes the project managers, the Active Directory stuff and so on, when you have such climate you must bend yourself and do your best.

I have the following scenario: couple of different Java apps deployed on Tomcats. Well that doesn’t sound so bad there are easy service wizard installs of Apache Tomcat on Windows, easier deployment of the same so I will not really bother with such a trivial things, I hope you can do that by yourself.

Where is the problem? Well there a plural applications and one open port for the outside world. That’s right, the standard HTTP 80 port.

Simple thing we should do is to port forward and url rewrite the addressing applications.

Before finding this solution I was aware of one way on connecting Java container with IIS, through ISAPI and JK. But that requires changing Windows registry, defining couple of configuration files, using isapi_redirect , in other words messy and hardly-working solution. There are couple of guides on the net, just google it if you want to try them. The next solution is clean, simple and productive.

The IIS I’m working on is version 7.5. That is nice because the fellow Microsoft guys introduce a new solution to us, ARR. The use of this module here is to be set up to work as proxy server and that is exactly what we need in this situation.

After installing this extension on the server machine (I’m working on Windows Server 2008) next thing is to enable proxy server. Follow this steps:

  1. Open the IIS Manager and in the IIS Group there should be Application Request Routing Cache module showing up. Select it.
  2. So here we are. In the Action menu choose – Server Proxy Settings .
  3. For the following scenario this set up will work : Enable proxy = true, checked keep-alive, time-out by will, reverse rewrite host headers in response checked. Apply the changes and proceed.

Half work done. Next thing that we need is the URL rewrite feature. Install it. (Here I don’t mention¬† anything about the features and roles that need to be installed to make IIS working. I think that is just not-needed information here. You have the wizards in Computer Management, use them).

Well I hope you got URL rewrite working. That’s where we will define the rewrites.

Lets imagine. Two  Tomcat installation with just one application per server.

Installation folders: D:\Tomcat1 , D:\Tomcat2 . We need only the HTTP connectors, so in $tomcat_home\conf\server.xml you can comment out the AJP connectors.

Don’t forget while installing the Tomcats, to configure them on different ports. Example :

¬† <Connector port=”8080″ protocol=”HTTP/1.1″ proxyName=”www.example.com” proxyPort=”80″
¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬† connectionTimeout=”20000″
¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬† redirectPort=”8443″ />

¬† <Connector port=”8081″ protocol=”HTTP/1.1″ proxyName=”www.example.com” proxyPort=”80″
¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬† connectionTimeout=”20000″
¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬† redirectPort=”8443″ />

You ask yourself why I have put proxyName and proxyPort here. Well they are not required but since we want this application to work for outside world as I mentioned , you don’t want Java calls like getServerName() to return localhost¬† and local port. We want ajax and applets to work. This way we ensure the right host name and port are returned.

Installation ports: Tomcat1 on port 8080, Tomcat2 on port 8081. Installed apps: on Tomcat1 app1, on Tomcat2 app2.

So we can open the deployed applications on following urls : http://localhost:8080/app1 and http://localhost:8081/app2 .

It’s time to finalize the installations. We want the apps to be accessible on the following urls http://www.example.com/app1 and http://www.example.com/app2 .

This is where URL rewrite comes in play and that’s where we do the things explicitly. You can define URL rewrite rules in the main configuration but I prefer creating new site bound to port 80 and defining it there. Add new site with name exampe.com and for physical path put D:\ or whatever you want for your local situation, this is just example. That’s the place where web.config file will be created and everything that we do in the IIS manager will be mapped in configuration file, which is an xml configuration file the IIS use.

  1. In the new site, IIS group, open URL rewrite.
  2. Add new rule from the actions menu. Choose blank rule.
  3. Configurations : Name РTomcat1 rule,  requested url -> matches the pattern , simple pattern: (app1.+) , ignore cases true,  action type Рrewrite , rewrite url: http://localhost:8080/{R:0}
  4. Apply , make another rule for the other tomcat and  just change the name, pattern and the port.
  5. Restart IIS and you got it.

Just to get things straight. URL rewrite allow us to filter by some rule patterns, conditions and replace server variables. We just defined simple rewrite rule. That’s why we use {R:0} . It simple variable for the first rule pattern. There can be more of course , the format is {R:N} , where N is from 0 to 9.

This was very simple. I hope this will give you some clues how can you manipulate this modules to do more complex stuff. And you are not limited to use only Java servers of course. Any socket listening on any port can work.

I wish you happy days, for the end a reward from me