Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Problem with DUS step #52

Open
JavierGorri opened this issue May 8, 2024 · 43 comments
Open

Problem with DUS step #52

JavierGorri opened this issue May 8, 2024 · 43 comments

Comments

@JavierGorri
Copy link

JavierGorri commented May 8, 2024 •
edited
Loading

Dear Theresa,

We have a problem when we try to run the R code of tha DUS step. Can you help us with it?

This is the trace:

rlang::last_trace()
<error/rlang_error>
Error in dplyr::collect():
! Failed to collect lazy table.
Caused by error:
! nanodbc/nanodbc.cpp:3108: 07009: [Microsoft][SQL Server Native Client 11.0]Invalid Descriptor Index

Backtrace:
▆

  1. ├─base::source("C:/Users/D229463/Desktop/Scripts_R/MegaStudy-Fase2/MegaStudy-main/DUS Code/CodeToRun.R")
  2. │ ├─base::withVisible(eval(ei, envir))
  3. │ └─base::eval(ei, envir)
  4. │ └─base::eval(ei, envir)
  5. ├─base::source(here("DUS.R")) at MegaStudy-main/DUS Code/CodeToRun.R:47:1
  6. │ ├─base::withVisible(eval(ei, envir))
  7. │ └─base::eval(ei, envir)
  8. │ └─base::eval(ei, envir)
  9. ├─base::source(here("individual_indications.R")) at MegaStudy-main/DUS Code/DUS.R:45:1
  10. │ ├─base::withVisible(eval(ei, envir))
  11. │ └─base::eval(ei, envir)
  12. │ └─base::eval(ei, envir)
  13. ├─CodelistGenerator::getCandidateCodes(...) at MegaStudy-main/DUS Code/individual_indications.R:3:1
  14. │ └─base::lapply(...)
  15. │ └─CodelistGenerator (local) FUN(X[[i]], ...)
  16. │ └─CodelistGenerator:::runSearch(...)
  17. │ └─CodelistGenerator:::getMatches(words = tidyWords(keywords), conceptDf = workingConcept)
  18. │ └─workingConcepts %>% dplyr::collect()
  19. ├─dplyr::collect(.)
  20. └─omopgenerics:::collect.cdm_table(.)
  21. ├─dplyr::collect(removeClass(x, "cdm_table"))
  22. └─dbplyr:::collect.tbl_sql(removeClass(x, "cdm_table"))
    Run rlang::last_trace(drop = FALSE) to see 6 hidden frames.

We find this.

image

These are the libraries a picture from the Console.

image

Kind regards,
@tiozab
Copy link
Collaborator

tiozab commented May 8, 2024

@JavierGorri, thanks I see you have sql_server, did you successfully run the renv for the DUS code?
packageVersion("CodelistGenerator")

In the outputfolder, as a default we call it "storage", there should be a log file, can you post this?
Did you do any changes in the code files?

@JavierGorri
Copy link
Author

@tiozab
We attach the txt and csv files that we can find
snapshot_Bardena.csv
BARDENA_log.txt
This is the code to run
CodeToRun.txt

We have done this change in the code file
image

@tiozab
Copy link
Collaborator

tiozab commented May 9, 2024

@JavierGorri Thanks. It seems you cannot generate the indication concepts.
can you give me the package versions please by running these three lines and telling my what the output is?
packageVersion("CodelistGenerator")
packageVersion("omopgenerics")
packageVersion("CDMConnector")

with regards to the changes you made to the code in line 36/37, what happens if you do not outcomment these lines?

@JavierGorri
Copy link
Author

@tiozab.
This tha package version
image
This is what happens:
image

@JavierGorri
Copy link
Author

JavierGorri commented May 9, 2024 via email

@tiozab
Copy link
Collaborator

tiozab commented May 9, 2024

Thanks @JavierGorri it seems like the use of renv has not worked, the renv.lock has CodelistGenerator 2.2.2 for example.
Please try again with renv::activate() and renv::restore()

let's not worry right now about the table names (i.e. use your outcommented versions). but first, have the correct package versions :-)

@JavierGorri
Copy link
Author

@tiozab
We have done it, This is the Error on a png
Error_R
image
When we see the renv.lock file it's yet on 2.2.3 version of the CodelistGenerator
image

Here we attach the documents of snapshot &logs:
BARDENA_log.txt
snapshot_Bardena.csv

@tiozab
Copy link
Collaborator

tiozab commented May 10, 2024

the codelistgenerator should be 2.2.2 :-)
can you pull again from the repository?
image

@JavierGorri
Copy link
Author

@tiozab We have begun to change to the correct package versions (some of then we had more recent ones). We have aproblem with the CirceR that we can not install.
image
When we try to install from the file we have this faile:
image
Can you help us with this?

@tiozab
Copy link
Collaborator

tiozab commented May 15, 2024

@JavierGorri you can use the circeR from CRAN (has been recently put on CRAN)
install.packages("CirceR")
hope that fixes it

@JavierGorri
Copy link
Author

Bad luck. We have this problem.
image

@tiozab
Copy link
Collaborator

tiozab commented May 15, 2024

https://cran.r-project.org/web//packages/CirceR/index.html
the package imports rJAVA package which seems to cause the problem.
Maybe by installing that yourselves before you can avoid the problem?
install.packages("rJava")

@JavierGorri
Copy link
Author

We have this error

install.packages("rJava")
The following package(s) will be installed:

  • rJava [1.0-11]
    These packages will be installed into "C:/Users/d229463/Desktop/EHDEN/MegaStudy-main/DUS Code/renv/library/windows/R-4.4/x86_64-w64-mingw32".

Do you want to proceed? [Y/n]: y

Installing packages --------------------------------------------------------

  • Installing rJava ... OK [linked from cache]
    Successfully installed 1 package in 23 milliseconds.

library("rJava")
Error: package or namespace load failed for ‘rJava’:
.onLoad falló en loadNamespace() para 'rJava', detalles:
llamada: inDL(x, as.logical(local), as.logical(now), ...)
error: no es posible cargar el objeto compartido 'C:/Users/d229463/AppData/Local/R/cache/R/renv/cache/v5/windows/R-4.4/x86_64-w64-mingw32/rJava/1.0-11/524167e771b81342f8dd7a891ed3afa8/rJava/libs/x64/rJava.dll':
LoadLibrary failure: %1 no es una aplicación Win32 válida.

@JavierGorri
Copy link
Author

@tiozab.
Hello, Theresa.

Do you have any Idea abouyt what we can do. We are really desperate.

Kind regards

@tiozab
Copy link
Collaborator

tiozab commented May 22, 2024

@janblom just gave this advice to another data partner concerning rJava / JAVA problems
• the version of java in terms of 32/64 bits should match your R version. You can check your R version by typing the command version in Rstudio. The second output line starting with "arch" should indicate whether it's 32 or 64. The screenshot below if not from a Windows environment, so it might not match what you see exactly, but there should be a clear 32/64 indication on the "arch" line. The java version you use should match that.

• The error mentioning "JAVA_HOME" indicates that the rJava package is not able to locate java in (any of) its default location(s). You can set the environment variable JAVA_HOME to point to the java installation directory, for example, if java is installed in C:\Program Files\Eclipse Adoptium\jdk-17.0.10.7-hotspot (with java.exe located here: C:\Program Files\Eclipse Adoptium\jdk-17.0.10.7-hotspot\bin\java.exe, you can set the environment variable like this (in Rstudio):
Sys.setenv(JAVA_HOME = "C:\Program Files\Eclipse Adoptium\jdk-17.0.10.7-hotspot")
Please change the directory to what applies in your environment,
Note that this only sets the environment variable in your current R/Rstudio session, you would need to repeat that every time you start R/Rstudio. It is usually better to set this permanently for the (Windows) user. I cannot provide you details on how to do that in your environment, if you do not know how to do that it is best to consult a local system support/admin person.

@janblom
Copy link

janblom commented May 27, 2024

Hi @JavierGorri,

did you manage to resolve this yet? If not, I think it does make sense to verify that both your R/Rstudio environment and Java installation match in terms of their binary compatibility, i.e. both should be either 32-bits, or 64-bits. If you need help verifying this, please let me know.

Best regards,
Jan Blom

@JavierGorri
Copy link
Author

JavierGorri commented May 27, 2024 via email

@JavierGorri
Copy link
Author

JavierGorri commented May 29, 2024 via email

@tiozab
Copy link
Collaborator

tiozab commented May 29, 2024

@JavierGorri omopgenerics should be 0.0.2

@JavierGorri
Copy link
Author

JavierGorri commented May 29, 2024 via email

@JavierGorri
Copy link
Author

JavierGorri commented May 29, 2024 via email

@tiozab
Copy link
Collaborator

tiozab commented May 29, 2024

do you want to try again to comment it out like you did before?
image

@JavierGorri
Copy link
Author

JavierGorri commented May 29, 2024 via email

@tiozab
Copy link
Collaborator

tiozab commented Jun 3, 2024

@JavierGorri I am sorry, I cannot see anything, can you send a screen shot and separately the log.txt from the storage folder? sorry that this is taking so long.

@JavierGorri
Copy link
Author

JavierGorri commented Jun 4, 2024 via email

@tiozab
Copy link
Collaborator

tiozab commented Jun 7, 2024

@JavierGorri I am not sure what the problem is, we can have a call this afternoon or monday afternoon

@JavierGorri
Copy link
Author

JavierGorri commented Jun 10, 2024 via email

@tiozab
Copy link
Collaborator

tiozab commented Jun 10, 2024

@JavierGorri I am not available thursday friday this week, however available again from 24/25th of June

@JavierGorri
Copy link
Author

JavierGorri commented Jun 11, 2024 via email

@tiozab
Copy link
Collaborator

tiozab commented Jun 18, 2024

@JavierGorri is it possible for you to change all table names and column names to lower case? do you not have your table names in lower case already because you have duplicates e.g. "Person" and "person"?
The problem that we see may be that the newer CDM Connector version does not allow your work around any more.

However, you can still try in the mean time a newer version that I wrote for some data partners who had problems with the DUS step (that could be fixed by newer package versions). However, as usual the correct use of the renv is very important.
DUS_1point4.zip

If you can have answers to these questions and a run of the newer version ready for our discussion on monday that will speed things up hopefully :-)

@JavierGorri
Copy link
Author

JavierGorri commented Jun 19, 2024 via email

@tiozab
Copy link
Collaborator

tiozab commented Jun 20, 2024

@JavierGorri I am sorry I seem not to be able to see your response since your last two comments, but only the communication that I wrote, can you have a look in github.com, at what it looks like, so you may be able to understand my problem with viewing your answer? That would facilitate our communication better :-)

@JavierGorri
Copy link
Author

JavierGorri commented Jun 20, 2024 via email

@tiozab
Copy link
Collaborator

tiozab commented Jun 24, 2024

@JavierGorri thanks for the call. I think it would make sense to add the github names of Fernando and Asier, so they can be part of the conversation :-)

@JavierGorri
Copy link
Author

Yes, a great Idea. How can I do it?

These are their emails [email protected] and [email protected]

@tiozab
Copy link
Collaborator

tiozab commented Jun 24, 2024

@D229463
@AsierBall

I keep you posted about the CDM requirements.

@JavierGorri
Copy link
Author

Thanks

@tiozab
Copy link
Collaborator

tiozab commented Jun 25, 2024

@JavierGorri @D229463 I just spoke with @edward-burn and he confirmed that all omop tables have to have lower case names and column names. Furthermore, it needs to contain all columns listed in the https://ohdsi.github.io/CommonDataModel/index.html 5.3 or 5.4.
The order of the columns does not matter and there can be your own additional columns within the omop tables.
Thanks for re-naming your data accordingly :-)

@JavierGorri
Copy link
Author

@tiozab Many many Thanks. We are working on it. Great help.

@tiozab
Copy link
Collaborator

tiozab commented Jun 28, 2024

https://ehden.sharepoint.com/:w:/r/sites/EHDENMEGA/Gedeelde%20documenten/General/Short%20descriptions%20of%20each%20database.docx?d=w3b7774a809374c4c84b226395c6e1660&csf=1&web=1&e=Vi3Kxh may I ask you to add your database description please (the document is in EHDEN Teams) thanks :-) @JavierGorri

@JavierGorri
Copy link
Author

JavierGorri commented Jul 1, 2024 via email

@JavierGorri
Copy link
Author

JavierGorri commented Jul 1, 2024 •
edited
Loading

We have change the column names and we have change the driver to ‘ODBC Driver 17 for SQL Server’ (we had used ‘SQL Server Native Client 11.0’) and we can run part of the code.
Thanks for helping us.

@tiozab
Copy link
Collaborator

tiozab commented Jul 3, 2024

I close this so we can continue on the new issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@janblom @JavierGorri @tiozab