Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

issue when using a volume for data #35

Closed
eriknl1982 opened this issue Apr 2, 2020 · 6 comments
Closed

issue when using a volume for data #35

eriknl1982 opened this issue Apr 2, 2020 · 6 comments

Comments

@eriknl1982
Copy link

eriknl1982 commented Apr 2, 2020 •
edited
Loading

Hi,

I have an issue with the jena-fuseki docker image.

I have the following configuration in my docker-compose file:

jena-fuseki:
image: stain/jena-fuseki:latest
expose:
- 3030
environment:
ADMIN_PASSWORD: ${JENA_FUSEKI_ADMIN_PASSWORD}
volumes:
- jena-fuseki_data:/fuseki

When I start this container for the first time, it works without any issues. When I stop the container and start it for a second time, I get the following errors in the log:

jena-fuseki_1 | [2020-04-02 11:11:25] TDB WARN Your platform does not support checking process liveness so TDB disk locations cannot be reliably locked to prevent possible corruption due to unsafe multi-JVM usage
jena-fuseki_1 | java.io.IOException: Cannot run program "ps": error=2, No such file or directory
jena-fuseki_1 | at java.base/java.lang.ProcessBuilder.start(Unknown Source)
jena-fuseki_1 | at java.base/java.lang.ProcessBuilder.start(Unknown Source)
jena-fuseki_1 | at org.apache.jena.tdb.sys.ProcessUtils.getProcessInfo(ProcessUtils.java:144)
jena-fuseki_1 | at org.apache.jena.tdb.sys.ProcessUtils.isAlive(ProcessUtils.java:85)
jena-fuseki_1 | at org.apache.jena.tdb.base.file.LocationLock.canObtain(LocationLock.java:148)
jena-fuseki_1 | at org.apache.jena.tdb.StoreConnection._makeAndCache(StoreConnection.java:280)
jena-fuseki_1 | at org.apache.jena.tdb.StoreConnection.make(StoreConnection.java:244)
jena-fuseki_1 | at org.apache.jena.fuseki.webapp.SystemState.init$(SystemState.java:89)
jena-fuseki_1 | at org.apache.jena.fuseki.webapp.SystemState.init(SystemState.java:56)
jena-fuseki_1 | at org.apache.jena.fuseki.webapp.SystemState.getDataset(SystemState.java:45)
jena-fuseki_1 | at org.apache.jena.fuseki.webapp.FusekiWebapp.initializeDataAccessPoints(FusekiWebapp.java:217)
jena-fuseki_1 | at org.apache.jena.fuseki.webapp.FusekiServerListener.serverInitialization(FusekiServerListener.java:93)
jena-fuseki_1 | at org.apache.jena.fuseki.webapp.FusekiServerListener.contextInitialized(FusekiServerListener.java:53)
jena-fuseki_1 | at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:952)
jena-fuseki_1 | at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:558)
jena-fuseki_1 | at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:917)
jena-fuseki_1 | at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:370)
jena-fuseki_1 | at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1497)
jena-fuseki_1 | at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1459)
jena-fuseki_1 | at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:847)
jena-fuseki_1 | at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:287)
jena-fuseki_1 | at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:545)
jena-fuseki_1 | at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
jena-fuseki_1 | at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:138)
jena-fuseki_1 | at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:108)
jena-fuseki_1 | at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
jena-fuseki_1 | at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:410)
jena-fuseki_1 | at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
jena-fuseki_1 | at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:138)
jena-fuseki_1 | at org.eclipse.jetty.server.Server.start(Server.java:416)
jena-fuseki_1 | at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:108)
jena-fuseki_1 | at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
jena-fuseki_1 | at org.eclipse.jetty.server.Server.doStart(Server.java:383)
jena-fuseki_1 | at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
jena-fuseki_1 | at org.apache.jena.fuseki.cmd.JettyFusekiWebapp.start(JettyFusekiWebapp.java:138)
jena-fuseki_1 | at org.apache.jena.fuseki.cmd.FusekiCmd.runFuseki(FusekiCmd.java:372)
jena-fuseki_1 | at org.apache.jena.fuseki.cmd.FusekiCmd$FusekiCmdInner.exec(FusekiCmd.java:356)
jena-fuseki_1 | at jena.cmd.CmdMain.mainMethod(CmdMain.java:93)
jena-fuseki_1 | at jena.cmd.CmdMain.mainRun(CmdMain.java:58)
jena-fuseki_1 | at jena.cmd.CmdMain.mainRun(CmdMain.java:45)
jena-fuseki_1 | at org.apache.jena.fuseki.cmd.FusekiCmd$FusekiCmdInner.innerMain(FusekiCmd.java:104)
jena-fuseki_1 | at org.apache.jena.fuseki.cmd.FusekiCmd.main(FusekiCmd.java:67)
jena-fuseki_1 | Caused by: java.io.IOException: error=2, No such file or directory
jena-fuseki_1 | at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
jena-fuseki_1 | at java.base/java.lang.ProcessImpl.(Unknown Source)
jena-fuseki_1 | at java.base/java.lang.ProcessImpl.start(Unknown Source)
jena-fuseki_1 | ... 42 more
jena-fuseki_1 | [2020-04-02 11:11:25] Server ERROR Exception in initialization: Process ID 8 can't open database at location /fuseki/system/ because it is already locked by the process with PID 7. TDB databases do not permit concurrent usage across JVMs so in order to prevent possible data corruption you cannot open this location from the JVM that does not own the lock for the dataset
jena-fuseki_1 | [2020-04-02 11:11:25] WebAppContext WARN Failed startup of context o.e.j.w.WebAppContext@4e61e4c2{Apache Jena Fuseki Server,/,file:///jena-fuseki/webapp/,UNAVAILABLE}
jena-fuseki_1 | org.apache.jena.tdb.TDBException: Process ID 8 can't open database at location /fuseki/system/ because it is already locked by the process with PID 7. TDB databases do not permit concurrent usage across JVMs so in order to prevent possible data corruption you cannot open this location from the JVM that does not own the lock for the dataset
jena-fuseki_1 | at org.apache.jena.tdb.StoreConnection._makeAndCache(StoreConnection.java:282)
jena-fuseki_1 | at org.apache.jena.tdb.StoreConnection.make(StoreConnection.java:244)
jena-fuseki_1 | at org.apache.jena.fuseki.webapp.SystemState.init$(SystemState.java:89)
jena-fuseki_1 | at org.apache.jena.fuseki.webapp.SystemState.init(SystemState.java:56)
jena-fuseki_1 | at org.apache.jena.fuseki.webapp.SystemState.getDataset(SystemState.java:45)
jena-fuseki_1 | at org.apache.jena.fuseki.webapp.FusekiWebapp.initializeDataAccessPoints(FusekiWebapp.java:217)
jena-fuseki_1 | at org.apache.jena.fuseki.webapp.FusekiServerListener.serverInitialization(FusekiServerListener.java:93)
jena-fuseki_1 | at org.apache.jena.fuseki.webapp.FusekiServerListener.contextInitialized(FusekiServerListener.java:53)
jena-fuseki_1 | at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:952)
jena-fuseki_1 | at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:558)
jena-fuseki_1 | at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:917)
jena-fuseki_1 | at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:370)
jena-fuseki_1 | at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1497)
jena-fuseki_1 | at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1459)
jena-fuseki_1 | at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:847)
jena-fuseki_1 | at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:287)
jena-fuseki_1 | at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:545)
jena-fuseki_1 | at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
jena-fuseki_1 | at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:138)
jena-fuseki_1 | at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:108)
jena-fuseki_1 | at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
jena-fuseki_1 | at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:410)
jena-fuseki_1 | at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
jena-fuseki_1 | at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:138)
jena-fuseki_1 | at org.eclipse.jetty.server.Server.start(Server.java:416)
jena-fuseki_1 | at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:108)
jena-fuseki_1 | at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
jena-fuseki_1 | at org.eclipse.jetty.server.Server.doStart(Server.java:383)
jena-fuseki_1 | at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
jena-fuseki_1 | at org.apache.jena.fuseki.cmd.JettyFusekiWebapp.start(JettyFusekiWebapp.java:138)
jena-fuseki_1 | at org.apache.jena.fuseki.cmd.FusekiCmd.runFuseki(FusekiCmd.java:372)
jena-fuseki_1 | at org.apache.jena.fuseki.cmd.FusekiCmd$FusekiCmdInner.exec(FusekiCmd.java:356)
jena-fuseki_1 | at jena.cmd.CmdMain.mainMethod(CmdMain.java:93)
jena-fuseki_1 | at jena.cmd.CmdMain.mainRun(CmdMain.java:58)
jena-fuseki_1 | at jena.cmd.CmdMain.mainRun(CmdMain.java:45)
jena-fuseki_1 | at org.apache.jena.fuseki.cmd.FusekiCmd$FusekiCmdInner.innerMain(FusekiCmd.java:104)
jena-fuseki_1 | at org.apache.jena.fuseki.cmd.FusekiCmd.main(FusekiCmd.java:67)
jena-fuseki_1 | [2020-04-02 11:11:25] Server INFO Started 2020/04/02 11:11:25 UTC on port 3030

the problem seems to be the file /fuseki/system/tdb.lock. Whenever this file is present, the above error is triggered. When I remove the file, the container starts normally again.

it seems to me that on shutdown of the container, this file should be cleaned up.
Is there any fix or workaround for this issue?
@dpnevmatikos
Copy link

dpnevmatikos commented Apr 3, 2020 •
edited
Loading

Hi,
Same issue here.

  1. Created the container with :
docker run  -p 3030:3030 -e ADMIN_PASSWORD=pw123 stain/jena-fuseki
  1. Uploaded a new dataset (ttl file)
  2. Stopped the container with: docker stop <name>
  3. Started the container with docker start <name>

and the issue appeared:

[2020-04-03 09:27:09] Server     ERROR Exception in initialization: caught: Process ID 7 can't open database at location /fuseki/databases/soda/ because it is already locked by the process with PID 8. TDB databases do not permit concurrent usage across JVMs so in order to prevent possible data corruption you cannot open this location from the JVM that does not own the lock for the dataset
[2020-04-03 09:27:09] WebAppContext WARN  Failed startup of context o.e.j.w.WebAppContext@329a1f8d{Apache Jena Fuseki Server,/,file:///jena-fuseki/webapp/,UNAVAILABLE}
org.apache.jena.assembler.exceptions.AssemblerException: caught: Process ID 7 can't open database at location /fuseki/databases/soda/ because it is already locked by the process with PID 8. TDB databases do not permit concurrent usage across JVMs so in order to prevent possible data corruption you cannot open this location from the JVM that does not own the lock for the dataset
  doing:
    root: http://base/#tdb_dataset_readwrite with type: http://jena.hpl.hp.com/2008/tdb#DatasetTDB assembler class: class org.apache.jena.tdb.assembler.DatasetAssemblerTDB

        at org.apache.jena.assembler.assemblers.AssemblerGroup$PlainAssemblerGroup.openBySpecificType(AssemblerGroup.java:165)
        at org.apache.jena.assembler.assemblers.AssemblerGroup$PlainAssemblerGroup.open(AssemblerGroup.java:144)
        at org.apache.jena.assembler.assemblers.AssemblerGroup$ExpandingAssemblerGroup.open(AssemblerGroup.java:93)
        at org.apache.jena.assembler.assemblers.AssemblerBase.open(AssemblerBase.java:39)
        at org.apache.jena.assembler.assemblers.AssemblerBase.open(AssemblerBase.java:35)
        at org.apache.jena.fuseki.build.FusekiConfig.getDataset(FusekiConfig.java:638)
        at org.apache.jena.fuseki.build.FusekiConfig.buildDataService(FusekiConfig.java:443)
        at org.apache.jena.fuseki.build.FusekiConfig.buildDataAccessPoint(FusekiConfig.java:433)
        at org.apache.jena.fuseki.build.FusekiConfig.readConfiguration(FusekiConfig.java:416)
        at org.apache.jena.fuseki.build.FusekiConfig.readConfigurationDirectory(FusekiConfig.java:395)
        at org.apache.jena.fuseki.webapp.FusekiWebapp.initializeDataAccessPoints(FusekiWebapp.java:216)
        at org.apache.jena.fuseki.webapp.FusekiServerListener.serverInitialization(FusekiServerListener.java:93)
        at org.apache.jena.fuseki.webapp.FusekiServerListener.contextInitialized(FusekiServerListener.java:53)
        at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:952)
        at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:558)
        at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:917)
        at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:370)
        at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1497)
        at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1459)
        at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:847)
        at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:287)
        at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:545)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:138)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:108)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
        at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:410)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:138)
        at org.eclipse.jetty.server.Server.start(Server.java:416)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:108)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
        at org.eclipse.jetty.server.Server.doStart(Server.java:383)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.apache.jena.fuseki.cmd.JettyFusekiWebapp.start(JettyFusekiWebapp.java:138)
        at org.apache.jena.fuseki.cmd.FusekiCmd.runFuseki(FusekiCmd.java:372)
        at org.apache.jena.fuseki.cmd.FusekiCmd$FusekiCmdInner.exec(FusekiCmd.java:356)
        at jena.cmd.CmdMain.mainMethod(CmdMain.java:93)
        at jena.cmd.CmdMain.mainRun(CmdMain.java:58)
        at jena.cmd.CmdMain.mainRun(CmdMain.java:45)
        at org.apache.jena.fuseki.cmd.FusekiCmd$FusekiCmdInner.innerMain(FusekiCmd.java:104)
        at org.apache.jena.fuseki.cmd.FusekiCmd.main(FusekiCmd.java:67)
Caused by: org.apache.jena.tdb.TDBException: Process ID 7 can't open database at location /fuseki/databases/soda/ because it is already locked by the process with PID 8. TDB databases do not permit concurrent usage across JVMs so in order to prevent possible data corruption you cannot open this location from the JVM that does not own the lock for the dataset
        at org.apache.jena.tdb.StoreConnection._makeAndCache(StoreConnection.java:282)
        at org.apache.jena.tdb.StoreConnection.make(StoreConnection.java:244)
        at org.apache.jena.tdb.StoreConnection.make(StoreConnection.java:258)
        at org.apache.jena.tdb.transaction.DatasetGraphTransaction.<init>(DatasetGraphTransaction.java:69)
        at org.apache.jena.tdb.sys.TDBMaker.createDirect(TDBMaker.java:126)
        at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(Unknown Source)
        at org.apache.jena.tdb.sys.TDBMaker._create(TDBMaker.java:112)
        at org.apache.jena.tdb.sys.TDBMaker.createDatasetGraphTransaction(TDBMaker.java:43)
        at org.apache.jena.tdb.TDBFactory._createDatasetGraph(TDBFactory.java:93)
        at org.apache.jena.tdb.TDBFactory.createDatasetGraph(TDBFactory.java:71)
        at org.apache.jena.tdb.assembler.DatasetAssemblerTDB.make(DatasetAssemblerTDB.java:57)
        at org.apache.jena.tdb.assembler.DatasetAssemblerTDB.createDataset(DatasetAssemblerTDB.java:48)
        at org.apache.jena.sparql.core.assembler.DatasetAssembler.open(DatasetAssembler.java:43)
        at org.apache.jena.assembler.assemblers.AssemblerGroup$PlainAssemblerGroup.openBySpecificType(AssemblerGroup.java:157)
        ... 41 more
[2020-04-03 09:27:09] Server     INFO  Started 2020/04/03 09:27:09 UTC on port 3030

It appears that Jena server is not shutting down gracefully.
Am I doing something wrong?

Update: Just noticed this solution and it seems to work.

@stain
Copy link
Owner

stain commented Apr 3, 2020

Thanks, I am able to reproduce. In a sense there are two issues:

  • procps is suddenly needed by Jena, so should be installed in container. ps is part of POSIX so perhaps not too bad of Jena to assume its existence.
  • Fuseki JVM is not cleanly shut down from docker stop
    • Is this because docker-entrypoint.sh puts the JVM in the background with & and so the signal is not passed along?

The entrypoint currently also does the optional loading after server is started, which is why the exec goes into background..

@inquota
Copy link

inquota commented Apr 8, 2020

Thanks for your explanation!

Is there a way for hook into this proces with the docker-entrypoint.sh and remove thetdb.lock before the container goes down?

@namedgraph
Copy link

  • Is this because docker-entrypoint.sh puts the JVM in the background with & and so the signal is not passed along?

You can save the PID from $! after the background command. But then you'll have to manually monitor its state, and ideally trap kill signals in the entrypoint script and pass them to the background process.

We tried a similar approach but abandoned it due to the complexity:
https://github.com/AtomGraph/fuseki-docker/blob/fuseki-tdb/entrypoint.sh

@kinow kinow mentioned this issue May 9, 2020
Merged
@thibaud-sanchez
Copy link

thibaud-sanchez commented Aug 7, 2020 •
edited
Loading

This is the same issue as #41
This is a problem with missing ps java.io.IOException: Cannot run program "ps": error=2, No such file or directory

I fixed it by creating my own DockerFIle which install procps and now when I stop my container lock files are removed :

FROM stain/jena-fuseki

RUN apt-get update; \
    apt-get install -y --no-install-recommends procps

There is a PR waiting merge : #40

@sroze sroze mentioned this issue Oct 25, 2020
Merged
@sroze
Copy link

sroze commented Apr 12, 2021

@stain FYI this issue can be closed.

@kuzeko kuzeko closed this as completed Apr 16, 2021
Msch0150 added a commit to Msch0150/numishare that referenced this issue Oct 20, 2022
@Msch0150
Workaround
daa21d8 
Workaround
stain/jena-docker#35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
8 participants
@stain @kuzeko @sroze @namedgraph @dpnevmatikos @inquota @eriknl1982 @thibaud-sanchez