martes, septiembre 22, 2009

Feature 'http://apache.org/xml/features/xinclude' is not recognized.

Coming across this configuration error in Solr14 configuration in a Cloudera Hadoop 0.18.3 distributed environment, I still have seen it in other areas too.

It seems to happen quite usually that there are several XML parsing libraries on a certain java application's classpath, and not all of that support the same sets of tags/attributes. Furthermore, sometimes there are even different versions of the same library that clash! An example of this is an application using a library also used by tomcat and an application.

In our case, it's true that this solved the problem:

-Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl

This forces the JVM to use that class as a DocumentVuilderFactory. We had several parsers in the classpath and Java had to choose one! And this is done quite in a random, although determistic way. (Randomness is what one says when doesn't know how something happens).

but well, it's different if you need the library on the job, in the datanode, rather than in the master where you can play more. The namenode (jobtracker) dispatches and creates the jvm instances on the datanodes - so it's difficult (which means I haven't seen a nice way to do it yet) to pass jvmarg to the job tasks.

So, we've just... included it in the code.

System.setProperty(
"javax.xml.parsers.DocumentBuilderFactory",
"com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl" );


There was a JIRA issue that documented that.


2009-10-21 11:39:05,606 ERROR org.apache.solr.core.Config: Exception during parsing file: solrconfig.xml:javax.xml.parsers.ParserConfigurationException: Feature 'http://apache.org/xml/features/xinclude' is not recognized.
at org.apache.xerces.jaxp.DocumentBuilderFactoryImpl.newDocumentBuilder(Unknown Source)
at org.apache.solr.core.Config.(Config.java:110)
at org.apache.solr.core.SolrConfig.(SolrConfig.java:130)
at org.apache.solr.core.SolrConfig.(SolrConfig.java:87)
at org.apache.solr.core.SolrCore.(SolrCore.java:494)
at com.enormo.supernova.mapred.IndexerJob$OutputFormat.getRecordWriter(IndexerJob.java:101)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:293)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2210)

2009-10-21 11:39:05,646 ERROR com.enormo.supernova.mapred.IndexerJob: Can't create SOLR Embedded server
javax.xml.parsers.ParserConfigurationException: Feature 'http://apache.org/xml/features/xinclude' is not recognized.
at org.apache.xerces.jaxp.DocumentBuilderFactoryImpl.newDocumentBuilder(Unknown Source)
at org.apache.solr.core.Config.(Config.java:110)
at org.apache.solr.core.SolrConfig.(SolrConfig.java:130)
at org.apache.solr.core.SolrConfig.(SolrConfig.java:87)
at org.apache.solr.core.SolrCore.(SolrCore.java:494)
at com.enormo.supernova.mapred.IndexerJob$OutputFormat.getRecordWriter(IndexerJob.java:101)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:293)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2210)
2009-10-21 11:39:05,656 WARN org.apache.hadoop.mapred.TaskTracker: Error running child
java.lang.NullPointerException
at org.apache.hadoop.mapred.ReduceTask$3.collect(ReduceTask.java:300)
at com.enormo.supernova.mapred.IndexerJob$IndexerJobReduce.reduce(IndexerJob.java:213)
at com.enormo.supernova.mapred.IndexerJob$IndexerJobReduce.reduce(IndexerJob.java:206)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:318)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2210)

1 comentario:

Anónimo dijo...

Thanks for your post! it helps me! I've had the same problem testing Solr 3.6.x with SolrTestCaseJ4 classes :-)