SOLR indexing failed: unknown field 'workflowDocumentation' #9390

lmaylein · 2023-02-15T15:13:36Z

We are using Dataverse 5.12.1 with this solr schema.xml (which was the last released schema.xml before 5.13): https://github.com/IQSS/dataverse/releases/download/v5.11/schema.xml

One of our published records is not indexed.

Maybe this is a result of adding the "Computational Workflow Metadata Block"?

This is what I've found in the server.log:

javax.ejb.EJBException: Error from server at http://localhost:8983/solr/collection1: ERROR: [doc=dataset_10589] unknown field 'workflowDocumentation'
	at com.sun.ejb.containers.EJBContainerTransactionManager.processSystemException(EJBContainerTransactionManager.java:723)
	at com.sun.ejb.containers.EJBContainerTransactionManager.completeNewTx(EJBContainerTransactionManager.java:652)
	at com.sun.ejb.containers.EJBContainerTransactionManager.postInvokeTx(EJBContainerTransactionManager.java:482)
	at com.sun.ejb.containers.BaseContainer.postInvokeTx(BaseContainer.java:4601)
	at com.sun.ejb.containers.BaseContainer.postInvoke(BaseContainer.java:2134)
	at com.sun.ejb.containers.BaseContainer.postInvoke(BaseContainer.java:2104)
	at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:220)
	at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:90)
	at com.sun.proxy.$Proxy362.indexDataset(Unknown Source)
	at edu.harvard.iq.dataverse.search.__EJB31_Generated__IndexServiceBean__Intf____Bean__.indexDataset(Unknown Source)
	at edu.harvard.iq.dataverse.engine.command.impl.FinalizeDatasetPublicationCommand.onSuccess(FinalizeDatasetPublicationCommand.java:242)
	at edu.harvard.iq.dataverse.EjbDataverseEngine.completeCommand(EjbDataverseEngine.java:342)
	at edu.harvard.iq.dataverse.EjbDataverseEngine.submit(EjbDataverseEngine.java:279)
	at jdk.internal.reflect.GeneratedMethodAccessor1139.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:588)
	at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:408)
	at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4835)
	at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:665)
	at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:834)
	at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:615)
	at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
	at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:140)
	at jdk.internal.reflect.GeneratedMethodAccessor270.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:888)
	at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:833)
	at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:615)
	at org.jboss.weld.module.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:72)
	at org.jboss.weld.module.ejb.SessionBeanInterceptor.aroundInvoke(SessionBeanInterceptor.java:52)
	at jdk.internal.reflect.GeneratedMethodAccessor264.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:888)
	at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:833)
	at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:375)
	at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4807)
	at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4795)
	at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:212)
	at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:90)
	at com.sun.proxy.$Proxy304.submit(Unknown Source)
	at edu.harvard.iq.dataverse.__EJB31_Generated__EjbDataverseEngine__Intf____Bean__.submit(Unknown Source)
	at edu.harvard.iq.dataverse.DatasetServiceBean.callFinalizePublishCommandAsynchronously(DatasetServiceBean.java:977)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:588)
	at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:408)
	at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4835)
	at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:665)
	at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:834)
	at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:615)
	at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
	at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:140)
	at jdk.internal.reflect.GeneratedMethodAccessor270.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:888)
	at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:833)
	at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:615)
	at org.jboss.weld.module.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:81)
	at org.jboss.weld.module.ejb.SessionBeanInterceptor.aroundInvoke(SessionBeanInterceptor.java:52)
	at jdk.internal.reflect.GeneratedMethodAccessor264.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:888)
	at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:833)
	at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:375)
	at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4807)
	at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4795)
	at com.sun.ejb.containers.EjbAsyncTask.call(EjbAsyncTask.java:101)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://localhost:8983/solr/collection1: ERROR: [doc=dataset_10589] unknown field 'workflowDocumentation'
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:681)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:266)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:214)
	at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:106)
	at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:71)
	at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:85)
	at edu.harvard.iq.dataverse.search.IndexServiceBean.addOrUpdateDataset(IndexServiceBean.java:1321)
	at edu.harvard.iq.dataverse.search.IndexServiceBean.addOrUpdateDataset(IndexServiceBean.java:732)
	at edu.harvard.iq.dataverse.search.IndexServiceBean.indexDataset(IndexServiceBean.java:600)
	at jdk.internal.reflect.GeneratedMethodAccessor1837.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:588)
	at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:408)
	at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4835)
	at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:665)
	at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:834)
	at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:615)
	at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
	at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:140)
	at jdk.internal.reflect.GeneratedMethodAccessor270.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:888)
	at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:833)
	at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:615)
	at org.jboss.weld.module.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:72)
	at org.jboss.weld.module.ejb.SessionBeanInterceptor.aroundInvoke(SessionBeanInterceptor.java:52)
	at jdk.internal.reflect.GeneratedMethodAccessor264.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:888)
	at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:833)
	at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:375)
	at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4807)
	at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4795)
	at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:212)
	... 71 more
]]

How can we resolve this problem?

Thanks

The text was updated successfully, but these errors were encountered:

poikilotherm · 2023-02-15T15:30:48Z

Hey there!

The workflow metadata block was added in v5.12, but despite its experimental nature, it was included in schema.xml by default. With release 5.13 we rolled back that accidental addition.

Please see step 7 of the upgrade instructions:

Replace Solr schema.xml to allow multiple production locations and support for geospatial indexing to be used. See specific instructions below for those installations without custom metadata blocks (1a) and those with custom metadata blocks (1b).
Note: with this release support for indexing of the experimental workflow metadata block has been removed from the standard schema.xml. If you are using the workflow metadata block be sure to follow the instructions in step 7b) below to maintain support for indexing workflow metadata.

Below that the steps necessary are described to re-add the fields.

Sorry for the inconvenience!

lmaylein · 2023-02-15T15:44:28Z

@poikilotherm Thanks. And this fix mentioned in the release notes of v5.13 should also work for v5.12.1?

poikilotherm · 2023-02-16T06:30:55Z

Yes it should, but in 5.12.1 the schema.xml still had these fields... so should not be necessary?

lmaylein · 2023-02-16T07:46:41Z

Unfortunately, this doesn't fix the problem. The workflowDocumentation field is not affected by this change.

Same error message:

{"status":"ERROR","message":"Problem indexing datasets/10589: javax.ejb.EJBException: Error from server at http://localhost:8983/solr/collection1: ERROR: [doc=dataset_10589] unknown field 'workflowDocumentation' org.apache.solr.client.solrj.impl.HttpSolrClient.RemoteSolrException Error from server at http://localhost:8983/solr/collection1: ERROR: [doc=dataset_10589] unknown field 'workflowDocumentation' "}

poikilotherm · 2023-02-16T09:31:59Z

Can you run the schema script? Retrieve the fields via API and update the schema.xml on your own should fix this for good. (I have no idea why there would be a field missing from the 5.12.1 schema.xml, AFAIK people tested this...)

lmaylein · 2023-02-16T09:42:21Z

@poikilotherm Can you give me a hint where to find this script? https://github.com/IQSS/dataverse/releases/download/v5.13/schema.xml does not contain this field either (because of the roll back you mentioned earlier?).

poikilotherm · 2023-02-16T11:32:07Z

Looking at

dataverse/conf/solr/8.11.1/schema.xml

Line 266 in cf90431

    
           <field name="workflowDocumentation" type="text_en" multiValued="true" stored="true" indexed="true"/>

(which is from v5.12.1), the field was present, and only removed in 5.13.

Here's the explanation for updating your index and here's a direct link to the script. Hope this helps!

lmaylein · 2023-02-16T11:59:18Z

Many thanks. I had only looked for a new schema file on the release pages: https://github.com/IQSS/dataverse/releases/tag/v5.12 . I was not aware that I need/can update the Solr schema after importing new metadata blocks.

pdurbin mentioned this issue Feb 15, 2023

9224 - revert workflow metadata block in Solr schema #9225

Merged

lmaylein closed this as completed Feb 16, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SOLR indexing failed: unknown field 'workflowDocumentation' #9390

SOLR indexing failed: unknown field 'workflowDocumentation' #9390

lmaylein commented Feb 15, 2023

poikilotherm commented Feb 15, 2023

lmaylein commented Feb 15, 2023

poikilotherm commented Feb 16, 2023

lmaylein commented Feb 16, 2023

poikilotherm commented Feb 16, 2023

lmaylein commented Feb 16, 2023

poikilotherm commented Feb 16, 2023

lmaylein commented Feb 16, 2023

SOLR indexing failed: unknown field 'workflowDocumentation' #9390

SOLR indexing failed: unknown field 'workflowDocumentation' #9390

Comments

lmaylein commented Feb 15, 2023

poikilotherm commented Feb 15, 2023

lmaylein commented Feb 15, 2023

poikilotherm commented Feb 16, 2023

lmaylein commented Feb 16, 2023

poikilotherm commented Feb 16, 2023

lmaylein commented Feb 16, 2023

poikilotherm commented Feb 16, 2023

lmaylein commented Feb 16, 2023