tomcat6 - net.sf.hibernate.util.JDBCExceptionReporter - Connection org.postgresql.jdbc3g.Jdbc3gConnection is closed -

i facing problem on 1 of production environments. happens quite not @ same segments of application, it's pretty random.

the application deployed on environment similar others deployed on different machines, wich used same technologies , close settings, error occurs on 1 it's being difficult trying find out problem.

i must these errors have started appear recently, since several update changes:

application migrated windows linux

postgressql 8.2 9.2

tomcat 5 tomcat 6

all these changes have been done on rest of deployed applications have not experienced problem @ all.

the technologies used applications are:

hibernate 2.1.6

postgresql 9.2

tomcat 6.0.35

the configuration in context.xml is:

<resource name="jdbc/psa" auth="container" type="javax.sql.datasource"         driverclassname="org.postgresql.driver" url="jdbc:postgresql://localhost:5432/psa?compatible=7.4"         username="xxx" password="xxx" maxactive="100" maxidle="30"         maxwait="10000" factory="org.apache.tomcat.dbcp.dbcp.basicdatasourcefactory"         defaultautocommit="false" removeabandoned="true"         removeabandonedtimeout="60" logabandoned="true" validationquery="select 1"         testonborrow="true" testonreturn="true" testwhileidle="true"         timebetweenevictionrunsmillis="300000"/> 

notice eviction active trying resolve problems didn't result.

the flow of typical problem next:

1.- new hibernate session opened

2.- check before of query if session opened , connected

3.- after see on logs session opened valid, exception produced

--added

the complete stack trace of 1 of errors this:

29 03 2013 10:00:00 info easyapscheduler_worker-8 com.psa.accounting.eureca.logic.eurecanlaccountinglogic - -- process eureca_nl starts     29 03 2013 10:00:00 info easyapscheduler_worker-2 com.psa.accounting.eureca.logic.eurecascaccountinglogic - -- process eureca_sc starts     29 03 2013 10:00:00 debug easyapscheduler_worker-2 com.xeridia.persistence.servicelocator - session opened: net.sf.hibernate.impl.sessionimpl@22880d66. opened: true, connected: true     29 03 2013 10:00:00 debug easyapscheduler_worker-2 com.xeridia.persistence.servicelocator - [getconnection]|16|thread[easyapscheduler_worker-2,4,main]|     29 03 2013 10:00:00 debug easyapscheduler_worker-2 com.xeridia.persistence.servicelocator - [getconnection]|16|thread[easyapscheduler_worker-2,4,main]|com.easyap.invoice.persistence.invoicedaohibernate.<init>(invoicedaohibernate.java:364)     29 03 2013 10:00:00 error easyapscheduler_worker-8 net.sf.hibernate.util.jdbcexceptionreporter - connection org.postgresql.jdbc3g.jdbc3gconnection@1b0ca3f7 closed.     29 03 2013 10:00:00 debug easyapscheduler_worker-2 com.xeridia.persistence.servicelocator - [getconnection]|16|thread[easyapscheduler_worker-2,4,main]|com.psa.invoice.persistence.psainvoicedaohibernate.<init>(psainvoicedaohibernate.java:52)     29 03 2013 10:00:00 error easyapscheduler_worker-8 net.sf.hibernate.util.jdbcexceptionreporter - connection org.postgresql.jdbc3g.jdbc3gconnection@1b0ca3f7 closed.     29 03 2013 10:00:00 error easyapscheduler_worker-8 net.sf.hibernate.util.jdbcexceptionreporter - not execute query     java.sql.sqlexception: connection org.postgresql.jdbc3g.jdbc3gconnection@1b0ca3f7 closed.         @ org.apache.tomcat.dbcp.dbcp.delegatingconnection.checkopen(delegatingconnection.java:398)         @ org.apache.tomcat.dbcp.dbcp.delegatingconnection.preparestatement(delegatingconnection.java:279)         @ org.apache.tomcat.dbcp.dbcp.poolingdatasource$poolguardconnectionwrapper.preparestatement(poolingdatasource.java:313)         @ net.sf.hibernate.impl.batcherimpl.getpreparedstatement(batcherimpl.java:257)         @ net.sf.hibernate.impl.batcherimpl.getpreparedstatement(batcherimpl.java:232)         @ net.sf.hibernate.impl.batcherimpl.preparequerystatement(batcherimpl.java:65)         @ net.sf.hibernate.loader.loader.preparequerystatement(loader.java:779)         @ net.sf.hibernate.loader.loader.doquery(loader.java:265)         @ net.sf.hibernate.loader.loader.doqueryandinitializenonlazycollections(loader.java:133)         @ net.sf.hibernate.loader.loader.dolist(loader.java:1033)         @ net.sf.hibernate.loader.loader.list(loader.java:1024)         @ net.sf.hibernate.hql.querytranslator.list(querytranslator.java:854)         @ net.sf.hibernate.impl.sessionimpl.find(sessionimpl.java:1544)         @ net.sf.hibernate.impl.queryimpl.list(queryimpl.java:39)         @ com.xeridia.persistence.basedaohibernate.findbyquery(basedaohibernate.java:312)         @ com.xeridia.persistence.basedaohibernate.findbyquery(basedaohibernate.java:291)         @ com.psa.invoice.persistence.psainvoicedaohibernate.getaccountinginvoiceswithnoproblem(psainvoicedaohibernate.java:215)         @ com.psa.accounting.eureca.logic.eurecanlaccountinglogic.getaccountingdatafromdb(eurecanlaccountinglogic.java:469)         @ com.psa.accounting.eureca.logic.eurecanlaccountinglogic.process(eurecanlaccountinglogic.java:207)         @ com.psa.accounting.eureca.scheduler.croneurecanlaccounting.execute(croneurecanlaccounting.java:91)         @ org.quartz.core.jobrunshell.run(jobrunshell.java:203)         @ org.quartz.simpl.simplethreadpool$workerthread.run(simplethreadpool.java:520) 

the class manages hibernate sessions class called servicelocator. method getconnection() of class returns new connection , code next:

synchronized (session) {                 s = (session) session.get();                 if ((s == null)||(!s.isopen())) {                     s = null;                     s = getsessionfactory().opensession();                     if (s != null) {                         log.debug("session opened: " + s + ". opened: " + s.isopen() +                                 ", connected: " + s.isconnected());                         printthreaddataforclosedconnections("[getconnection]");                     }                     session.set(s);                 }                 if (!s.isconnected()) {                     s.reconnect();                     log.debug("session reconnected: " + s + ". opened: " + s.isopen() +                             ", connected: " + s.isconnected());                     printthreaddataforclosedconnections("[getconnection]");                 }             } 

the object 'session' declared as:

public static final threadlocal session = new threadlocal(); 

to obtain different sessions different threads, there shouldn't exist problem closing sessions between threads.

what can see in stack trace above easyapscheduler_worker-2 obtains new connection because of if ((s == null)||(!s.isopen())) { easyapscheduler_worker-8 not, means 'session' objects exists , open or fails obtain connection , corresponds first trace of error 29 03 2013 10:00:00 error easyapscheduler_worker-8 net.sf.hibernate.util.jdbcexceptionreporter - connection org.postgresql.jdbc3g.jdbc3gconnection@1b0ca3f7 closed.

i don't know if problem comes hibernate, postgres or tomcat. haven't found nothing error on google hope can give me hand.

please ask whatever information or settings need in order help.

thanks in advance.

i fixed this.

it problem quartz threads. default, quartz manages pool of threads re-used between executions of different jobs. class servicelocator, responsible open , close connections, based on threadlocal objects. seems 1 of schedule jobs didn't close connection properly, , after that, jobs executed on same thread 1 caused error.

i couldn't figure out problematic job, , did change default connection pool of quartz custom one, in such way make new thread each job execution, instead of sharing amount of threads jobs.

here quartz custom thread pooling have wrote in case useful someone:

public class simplenothreadpooling implements threadpool {      public int getpoolsize() {         return 0;     }       public log getlog() {         return logfactory.getlog(simplenothreadpooling.class);     }      public void initialize() throws schedulerconfigexception { }      class workerthread extends thread {          private runnable runnable = null;          /**          * <p>          * create worker thread, start it, execute runnable , terminate          * thread (one time execution).          * </p>          */         workerthread(runnable runnable) {             this.runnable = runnable;             start();         }          public void run() {             if (runnable != null) {                 try {                     runnable.run();                 } catch (exception exceptioninrunnable) {                     try {                         getlog().error("error while executing runnable: ",                             exceptioninrunnable);                     } catch(exception e) {                           // ignore tomcat glitch                     }                 }             }         }     }      public boolean runinthread(runnable runnable) {         new workerthread(runnable);         return true;     }      public void shutdown(boolean waitforjobstocomplete) { }  }