Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Project description links information is not reconciled on project close and reopen #470

Open
trancexpress opened this issue May 31, 2023 · 6 comments · May be fixed by #473
Open

Project description links information is not reconciled on project close and reopen #470

trancexpress opened this issue May 31, 2023 · 6 comments · May be fixed by #473
Labels
bug Something isn't working

Comments

@trancexpress
Copy link
Contributor

In our product, we have a builder that reads linked folders. The builder runs into a NPE, due to stale links information. The stale links information results due to the following sequence:

  1. Debug Eclipse with a breakpoint at the start of Project.reconcileLinksAndGroups().
  2. Create a Java project.
  3. Add a linked source folder to some folder on disk.
  4. Close the project.
  5. Delete the information about the link from the .project file on disk, e.g. delete the following block (location will be different):
    <linkedResources>
	    <link>
		    <name>test</name>
		    <type>2</type>
		    <location>/data/tmp/test</location>
	    </link>
    </linkedResources>
  1. Open the project in Eclipse.
  2. Observe oldDescription.getLinks() returns null and so nothing is done to refresh the links information.

The information about links in the project description is not persisted in LocalMetaArea.writePrivateDescription() and is also not read in LocalMetaArea.readPrivateDescription(). We should fix this, to ensure up-to-date links information in the project description after opening a closed project.

@iloveeclipse
Copy link
Member

5. Delete the information about the link from the .project file on disk, e.g. delete the following block

Just wondering, why should project still point to the link if this information is explicitly deleted from .project?
Do I miss something?

@trancexpress
Copy link
Contributor Author

Just wondering, why should project still point to the link if this information is explicitly deleted from .project? Do I miss something?

I don't think it should, but the link information is stale and not updated (this bug). Our product is using classpath information (the claspath for whatever reason remains the same despite the .project change), for a classpath entry the code asks the project if a folder is linked, the project replies with "yes" (this bug).

@trancexpress trancexpress changed the title Project description links information is not persisted by LocalMetaArea Project description links information is not reconciled on project close and reopen May 31, 2023
@trancexpress
Copy link
Contributor Author

trancexpress commented May 31, 2023

Seems like LocalMetaArea.readPrivateDescription() is for legacy state. It doesn't run any code now, it just exits early. Instead the description is read with this line:

description = new ProjectDescriptionReader(target).read(new InputSource(in));

I.e. it reads the current disk state.

The old description is supposed to be the one in memory. Unfortunately its no longer available at the time of the re-open.

After the re-open, a project description is created here:

"Worker-33: Open Project" #115 prio=5 os_prio=0 cpu=47.37ms elapsed=104.97s tid=0x00007ffe5c997f80 nid=0x10b48 at breakpoint [0x00007ffe9cef4000]
   java.lang.Thread.State: RUNNABLE
        at org.eclipse.core.internal.resources.ProjectDescription.<init>(ProjectDescription.java:107)
        at org.eclipse.core.internal.resources.ProjectDescriptionReader.startElement(ProjectDescriptionReader.java:990)
        at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement([email protected]/AbstractSAXParser.java:518)
        at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement([email protected]/XMLNSDocumentScannerImpl.java:374)
        at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl$NSContentDriver.scanRootElementHook([email protected]/XMLNSDocumentScannerImpl.java:613)
        at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next([email protected]/XMLDocumentFragmentScannerImpl.java:3079)
        at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next([email protected]/XMLDocumentScannerImpl.java:836)
        at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next([email protected]/XMLDocumentScannerImpl.java:605)
        at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next([email protected]/XMLNSDocumentScannerImpl.java:112)
        at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument([email protected]/XMLDocumentFragmentScannerImpl.java:542)
        at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse([email protected]/XML11Configuration.java:889)
        at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse([email protected]/XML11Configuration.java:825)
        at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse([email protected]/XMLParser.java:141)
        at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse([email protected]/AbstractSAXParser.java:1224)
        at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse([email protected]/SAXParserImpl.java:637)
        at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse([email protected]/SAXParserImpl.java:326)
        at org.eclipse.core.internal.resources.ProjectDescriptionReader.read(ProjectDescriptionReader.java:935)
        at org.eclipse.core.internal.localstore.FileSystemResourceManager.read(FileSystemResourceManager.java:897)
        at org.eclipse.core.internal.resources.SaveManager.restoreMetaInfo(SaveManager.java:894)
        at org.eclipse.core.internal.resources.SaveManager.restore(SaveManager.java:769)
        at org.eclipse.core.internal.resources.Project.open(Project.java:1067)
        at org.eclipse.ui.actions.OpenResourceAction$1.doOpenWithReferences(OpenResourceAction.java:233)
        at org.eclipse.ui.actions.OpenResourceAction$1.runInWorkspace(OpenResourceAction.java:279)
        at org.eclipse.core.internal.resources.InternalWorkspaceJob.run(InternalWorkspaceJob.java:43)
        at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)

Another description is then created with this call:

"Worker-31: Refreshing workspace" #111 prio=5 os_prio=0 cpu=91.29ms elapsed=867.35s tid=0x00007ffebc1aa000 nid=0x103df at breakpoint [0x00007ffe9f5f3000]
   java.lang.Thread.State: RUNNABLE
        at org.eclipse.core.internal.resources.ProjectDescription.<init>(ProjectDescription.java:107)
        at org.eclipse.core.internal.resources.ProjectDescriptionReader.startElement(ProjectDescriptionReader.java:990)
        at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement([email protected]/AbstractSAXParser.java:518)
        at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement([email protected]/XMLNSDocumentScannerImpl.java:374)
        at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl$NSContentDriver.scanRootElementHook([email protected]/XMLNSDocumentScannerImpl.java:613)
        at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next([email protected]/XMLDocumentFragmentScannerImpl.java:3079)
        at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next([email protected]/XMLDocumentScannerImpl.java:836)
        at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next([email protected]/XMLDocumentScannerImpl.java:605)
        at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next([email protected]/XMLNSDocumentScannerImpl.java:112)
        at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument([email protected]/XMLDocumentFragmentScannerImpl.java:542)
        at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse([email protected]/XML11Configuration.java:889)
        at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse([email protected]/XML11Configuration.java:825)
        at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse([email protected]/XMLParser.java:141)
        at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse([email protected]/AbstractSAXParser.java:1224)
        at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse([email protected]/SAXParserImpl.java:637)
        at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse([email protected]/SAXParserImpl.java:326)
        at org.eclipse.core.internal.resources.ProjectDescriptionReader.read(ProjectDescriptionReader.java:935)
        at org.eclipse.core.internal.localstore.FileSystemResourceManager.read(FileSystemResourceManager.java:897)
        at org.eclipse.core.internal.resources.Project.updateDescription(Project.java:1384)
        at org.eclipse.core.internal.resources.File.updateMetadataFiles(File.java:379)
        at org.eclipse.core.internal.localstore.RefreshLocalVisitor.visit(RefreshLocalVisitor.java:306)
        at org.eclipse.core.internal.localstore.UnifiedTree.accept(UnifiedTree.java:119)
        at org.eclipse.core.internal.localstore.FileSystemResourceManager.refreshResource(FileSystemResourceManager.java:978)
        at org.eclipse.core.internal.localstore.FileSystemResourceManager.refresh(FileSystemResourceManager.java:961)
        at org.eclipse.core.internal.resources.Resource.refreshLocal(Resource.java:1573)
        at org.eclipse.core.internal.refresh.RefreshJob.runInWorkspace(RefreshJob.java:228)
        at org.eclipse.core.internal.resources.InternalWorkspaceJob.run(InternalWorkspaceJob.java:43)
        at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)

So the new description is the last one, created when opening the project. And the previous (old) description is also read from disk when opening the project (and so has no links information), as opposed to using the in-memory description (that had links information).

The in-memory description is gone after the close, e.g. if I add this code, there is no old description at the time of this new reconcile call:

diff --git a/resources/bundles/org.eclipse.core.resources/src/org/eclipse/core/internal/resources/SaveManager.java b/resources/bundles/org.eclipse.core.resources/src/org/eclipse/core/internal/resources/SaveManager.java
index 24b0df8412..794183bbf3 100644
--- a/resources/bundles/org.eclipse.core.resources/src/org/eclipse/core/internal/resources/SaveManager.java
+++ b/resources/bundles/org.eclipse.core.resources/src/org/eclipse/core/internal/resources/SaveManager.java
@@ -908,6 +908,9 @@ public class SaveManager implements IElementInfoFlattener, IManager, IStringPool
                        //try to read private metadata and add to the description
                        workspace.getMetaArea().readPrivateDescription(project, description);
                }
+               if (project.isOpen()) {
+                       project.reconcileLinksAndGroups(description);
+               }
                project.internalSetDescription(description, false);
                if (failure != null) {
                        try {

@trancexpress
Copy link
Contributor Author

trancexpress commented May 31, 2023

Considering there is no old state to compare to, I wonder if we want to delete the linked folder resources (the project children, not the actual target folders) from the workspace on project close. If they are re-created on project open (due to being read from .project) then this is maybe OK to do... I did try to clear the linked flag tom the resource info of the linked folders but that somehow didn't work. Will have to double check why it doesn't help in our product.

Then again there already is this code in Project.internalClose():

		// remove each member from the resource tree.
		// DO NOT use resource.delete() as this will delete it from disk as well.
		IResource[] members = members(IContainer.INCLUDE_PHANTOMS | IContainer.INCLUDE_TEAM_PRIVATE_MEMBERS | IContainer.INCLUDE_HIDDEN);
		subMonitor.setWorkRemaining(members.length);
		for (IResource member2 : members) {
			Resource member = (Resource) member2;
			workspace.deleteResource(member);
			subMonitor.worked(1);
		}

I guess the next step here is to add a test, so that we are not depending on our product to reproduce. Making it easier to debug and understand what resource infos exist and why they are stale.

trancexpress added a commit to trancexpress/eclipse.platform that referenced this issue Jun 1, 2023
Fixes: eclipse-platform#470
Signed-off-by: Simeon Andreev <[email protected]>
trancexpress added a commit to trancexpress/eclipse.platform that referenced this issue Jun 1, 2023
Whenever a project is closed, its resource tree is saved. This includes
linked resources in the project. When the project is re-opened, link
changes in the .project file are not reflected on the projects resource
tree. The old resource tree is read, new information is stored in
ProjectDescription.linkDescriptions, but the old linked resources are
not touched.

This change adjusts Project.open() and Project.close() to set resp.
clear the M_LINK flag of linked resources in the project.

Fixes: eclipse-platform#470
Signed-off-by: Simeon Andreev <[email protected]>
iloveeclipse pushed a commit to trancexpress/eclipse.platform that referenced this issue Jun 14, 2023
Whenever a project is closed, its resource tree is saved. This includes
linked resources in the project. When the project is re-opened, link
changes in the .project file are not reflected on the projects resource
tree. The old resource tree is read, new information is stored in
ProjectDescription.linkDescriptions, but the old linked resources are
not touched.

This change adjusts Project.open() and Project.close() to set resp.
clear the M_LINK flag of linked resources in the project.

Fixes: eclipse-platform#470
Signed-off-by: Simeon Andreev <[email protected]>
@iloveeclipse
Copy link
Member

With the current state of affairs, deleting links from .project file by external file modification isn't supported and "leaks" any linked folder added before.

While adding "linked" folders one has provided a way to create them automatically (by reading .project file) but didn't provided a 100% safe way to delete them automatically - it works only if they were removed by Eclipse UI or by editing the file in Eclipse.

The use case not working is "external delete" by modifying .project file outside of Eclipse while project is closed or Eclipse is not open at all.

image

<?xml version="1.0" encoding="UTF-8"?>
<projectDescription>
	<name>A</name>
	<comment></comment>
	<projects>
	</projects>
	<buildSpec>
		<buildCommand>
			<name>org.eclipse.jdt.core.javabuilder</name>
			<arguments>
			</arguments>
		</buildCommand>
		<buildCommand>
			<name>edu.umd.cs.findbugs.plugin.eclipse.findbugsBuilder</name>
			<arguments>
			</arguments>
		</buildCommand>
	</buildSpec>
	<natures>
		<nature>org.eclipse.jdt.core.javanature</nature>
		<nature>edu.umd.cs.findbugs.plugin.eclipse.findbugsNature</nature>
	</natures>
<!-- this part has to be removed while project is closed or Eclipse not started -->
	<linkedResources>
		<link>
			<name>wsp_428</name>
			<type>2</type>
			<location>/tmp/wsp_428</location>
		</link>
	</linkedResources>
</projectDescription>

The "externally deleted" linked folder has following implications:

  • In Explorers, it is shown but with a warning overlay (org.eclipse.ui.internal.ide.LinkedResourceDecorator.decorate())
  • In Explorers, for linked folders, content (children) can't be seen.
  • org.eclipse.core.filesystem.IFileInfo says it does NOT exists
  • org.eclipse.core.resources.IResource.exists() says it exists

Looking at the code, I assume org.eclipse.core.internal.resources.Project.reconcileLinksAndGroups(ProjectDescription) should not only check content of the .project file but also consider to visit all the existing links and check them for corresponding entry in the .project file (== ProjectDescription.getLinks() match).

@iloveeclipse
Copy link
Member

I believe proposed PR doesn't cover the use case we have completely, as it only works on open/close of projects. So it can't detect changes made while Eclipse was closed (e.g. close Eclipse, change .project file to remove links, open Eclipse).

What we need is to detect external changes independently on open/close/not started Eclipse.
I believe the change should try to hook into the project restore operation which happens both on startup and project open
SaveManager.restore(Project, IProgressMonitor) and
SaveManager.restoreMetaInfo(Project, IProgressMonitor).

There we could try to use knowledge we have from .project timestamp (change should indicate links reconcile is needed)

  • FileSystemResourceManager.read(IProject, boolean)
  • FileSystemResourceManager.isDescriptionSynchronized(IProject)

To reconcile links info we can brute force traverse whole tree and check if the found links are known by ProjectDescription.getLinks().
However, the more elegant (and scalable) solution would be to store ProjectDescription.getLinks() info into the .metadata (new file?) and read it back on project restore, comparing old .metadata state vs found .project state. Later should win and all "extra" links existing in .metadata should be removed from the resource tree (via oldLinkResource.delete(IResource.NONE, null), similar how it is done in Project.reconcileLinksAndGroups(ProjectDescription)).

iloveeclipse pushed a commit to trancexpress/eclipse.platform that referenced this issue Apr 25, 2024
Whenever a project is closed, its resource tree is saved. This includes
linked resources in the project. When the project is re-opened, link
changes in the .project file are not reflected on the projects resource
tree. The old resource tree is read, new information is stored in
ProjectDescription.linkDescriptions, but the old linked resources are
not touched.

This change adjusts Project.open() and Project.close() to set resp.
clear the M_LINK flag of linked resources in the project.

Fixes: eclipse-platform#470
Signed-off-by: Simeon Andreev <[email protected]>
@iloveeclipse iloveeclipse added the bug Something isn't working label Apr 25, 2024
iloveeclipse added a commit to iloveeclipse/eclipse.platform that referenced this issue Apr 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants