Programming stuff: Bash, Linux, SQL and Java

we have a web application at work, that updates a web page comet style.

it is running on tomcat.

the update is done based on some events from a backend application, and the connections are kept open for a long time, untill the event triggers the update.

we had the 1 thread per connection situation and as the traffic grew, we had to make a change.

tomcat had an event based comet model, and i couldn’t make it work.

so i switched to jetty and its continuation thing.
in jetty, the principle is that an Exception is thrown, and then the connector catches it and frees the thread, retruning it to the pool so it can be reused for other requests.

But, my application was a struts one, and i had the suspend() call inside an Action.
after many tries, all my efforts made the examples i found on the net work perfectly, but my app didn’t.
all it did was to spit the retryrequest runtime exception in the web page.
finally , the trick was that ProcessRequest class in the struts stack was throwing all exceptions that occured in the Actions like this:
throw new ServletException(exception) , and the SelectChannelConnector in the jetty stack couldn’t get its hand on its retryRequest exception.
so i changed the ProcessRequest class, so that RetryRequest exception is skipped, and all was great.
so, in order to make jetty’s continuations work with struts stack you have to be aware of this RetryRequest  runtime exception to be thrown exactly as it is in the struts stack and not wrapped in the ServletException.

Comments(1)

if you want to remove class and method information line when you do logging with java.util.logging , then you have 2 choices:
1. subClass your handler and make your own publish method
or
2. subClass the formatter you pass to the handler and modify the format(LogRecord record) method.

i did the second one.
here it is:
public String format(LogRecord lr){
                           return lr.getMessage()+"\n";
                   }

Comments(0)

Today, at work, we faced insert and delete problems in our mysql database.
usually we have bulks of inserts/updates/deletes from a backend engine, and selects from the website.
the inserts/updates/deletes bulks are usually around 100-500, but at peak times it goes up to 40 000.
the application that does it is a java application and it does it with PreparedStatement and batch.
When we first did this, we thought it would be enough, and no further optimisation could occur.
To my surprise, there is more to it. instead of 5000 "insert into table values (?,?,?,?,?)", this proves to be 5 times faster, in our case:
"insert into table values(?,?,?,?,?),(?,?,?,?,?),(?,?,?,?,?),….5000 times..,(?,?,?,?,?)".
there is some time wasted in building the string, but it is much better this way, as the load on our servers is on the mysql engine, the rest of the applications are low.
you can find the source code of a java class that tests the difference between the 2 approaches, so you can test it at home.
the times are: ~ 1235 ms for the thousand of inserts in batch and 211 ms for the one big insert with 5000 values() pairs.
the stringbuilding of the sql takes ~ 300 ms which is good for our purposes.

1. note: for deleting, always use delete from table where id IN (?,?,?…..?)

2. for the source code file, i did copy from eclipse and paste in vim on the web-server. the code got autoindenting and from 1 comment it commented all the way down. here is the trick that saves you: :set paste before pasting and :set nopaste after.

Comments(0)

for my swing applications:
./configure –without-corba –without-rmi –without-imageio –without-management –without-naming
and in java6, –without-derby
also: –enable-opengl and all those desktop and swing optimisations availbale.

but i figure i have to wait before i can do this.

Comments(0)