Releases: rfhb/ctrdata
ctrdata v1.19.4
ctrdata v1.19.3
- Improve Tampermonkey script (this serves to copy searches in any register to the clipboard, and also can open a CTIS search from an URL, see https://rfhb.github.io/ctrdata/#id_2-script-to-automatically-copy-users-query-from-web-browser
- Refactored parts of documents downloading for handling CDN links
- Simplify code now that CTIS provides an advanced search
- Code styling and linting
ctrdata v1.19.2
Improvements
- Added typing for newly appearing variables in
CTIS
- Accelerated
CTIS
trial data processing
Bug fixes
- Fix incomplete downloads from
CTIS
by disabling HTTP/2 multiplexing in another function
ctrdata v1.19.1
Improvements
- Revised translation of location elements from search URL to API call for
CTGOV2
ctrdata v1.19.0
Possibly breaking changes
-
CTGOV
has retired on 2024-06-25 the classic website and API used byctrdata
since 2015. To support users,ctrdata
now automatically translates and redirects queries to the current website. This helps with automatically updating previously loaded queries (ctrLoadQueryIntoDb(querytoupdate = <n>)
), manually migrating queries and reproducible work on clinical trials information. This new functionality inctrdata
translates a user's search query URL from the classic website into a query for the currentCTGOV
website, for all search parameters. Since the structure and format of data differs between data retrieved from the current API and previously retrieved from the classic API,ctrdata
will continue to identify the current API asregister = "CTGOV2"
, to support the analysis stage. In addition,ctrdata
documentation continues to include examples of analyses withCTGOV
data, as this may have been downloaded earlier. -
CTIS
has been relaunched on 2024-06-17, andctrdata
has been fully updated to it. At the moment,CTIS
provides basic searches and no search query URL. To support users,ctrdata
includes an updated script that extracts a user's search parameters from the register search page to the clipboard and into the browser URL bar. In addition, the script triggers a search for trials when opening such a query URL, see https://rfhb.github.io/ctrdata/#id_2-script-to-automatically-copy-users-query-from-web-browser. File names ofCTIS
documents start now with the document type, e.g.SbjctInfaICF - L1 SIS and ICF NL for publication.pdf
, since the prefix (document source) introduced in version 1.18.0 is no more applicable.
Improvements
- Reduced size of demo database in package, addressing
CRAN
notes - Adapted
ctrFindActiveSubstanceSynonyms()
toCTGOV2
API; this is now based on terms used in studies - Updated vignette and added inspecting a specific trial
- Updated register information, adding caveats and issues
- Updated estimated download sizes and user information
- Avoided duplicate data and file retrievals
- New attribute
ctrdata-collection
for data frames with trial information; same content as attributectrdata-table
, which will be removed by end 2024 - Split utils.R into files for functions and fields
Bug fixes
- Correct partial date brackets for
CTGOV2
- Disable HTTP/2 multiplexing for
CTIS
ctrdata v1.18.0
Possibly breaking change
- File names for documents downloaded from
CTIS
now include the document type and use different separators (e.g., "parts2auth - SbjctInfaICF - ..." as abbreviation of "Subject information and informed consent form (for publication)", was previously "parts2auth_...")
Improvements
- Add retrieval of historic versions of trial records from CTGOV2 (automatically retrieved from CTIS, not available for other registers)
- Added typing of newly appearing fields from CTIS
- Increase testing coverage to 93.7% locally
- Added missing CI for the combination of DuckDB and CTGOV2
Bug fixes
- Correct typing certain fields as (lists of) integers
- Handle marginal case in
dbGetFieldsIntoDf()
- Changed
unlink
of intermediate files - Minor vignettes corrections
ctrdata v1.17.2
Bug fixes
- Switch sequence of API endpoints used with
CTIS
- Correct handling multiple public events with
CTIS
- Re-use
CTIS
downloads in a given session
ctrdata v1.17.1
Improvements
- Additional
CTIS
field types and possibly documents (associated clinical trials) - Use
ctId
instead ofid
asCTIS
top-level field (a further synonym used in the API isclinicalTrialId
)
Bug fixes
- Added handling of unexpected Unicode in protocol-related data from one trial in
EUCTR
- Improved predicted download sizes and times for
EUCTR
- Return with message if
CTIS
query did not result in any trials
ctrdata v1.17.0
Possibly breaking changes
- Reimplemented
dbGetFieldsIntoDf()
to accelerate and have more predictable, simplified returns, in particular for nested fields; also attempts to recursively expand simply nested data into additional columns in the returned data frame - Reimplemented
dbFindFields()
to accelerate; both based on improvednodbi::docdb_query()
dbFindFields()
now digests a sample of records to quickly find fields, or all records ifsample = FALSE
but taking increasing time with increasing number of records- If using
nodbi::scr_postgres()
, parameterfields
ofdbGetFieldsIntoDf()
is limited to less than 50 fields; a message flags for any backend potential compatibility issues, suggesting to use parent fields, e.g.,a.b
instead ofc("a.b.c.d", "a.b.c.e")
- Parameter
stopifnodata
ofdbGetFieldsIntoDf()
is no more needed and deprecated - Reimplemented typing fields to speed up and to simplify
Improvements
- Register data are re-used and not downloaded again in an interactive session (that is, the same temporary folder is now re-used throughout a user's session)
- Temporary folder can be set by users with
options(ctrdata.tempdir = "<user_specified_folder>")
- Inform MS Windows users if
cygwin
was found so that they may chose to delete it - Many fields added for typing e.g. as date in
dbGetFieldsIntoDf()
Bug fixes
- Adapted and corrected information loading to newly available data in
CTIS
- Corrected escaping, and back-conversion, of characters in
JSON
fromCTIS
ctrdata v1.16.0
Possibly breaking changes
XML files are converted slightly differently
- EUCTR result-related information in attributes: e.g. new:
{"id":"PostAssignmentPeriod-46349"}
, was:{"@attributes":{"id":"PostAssignmentPeriod-46349"}}
- Consequently, it should work to just delete
@attributes
from field names such asdbGetFieldsIntoDf("clinical_results.baseline.analyzed_list.analyzed.count_list.count.@attributes.value", db)
- EUCTR protocol-related information although no differences were found yet
- CTGOV attributes of bare values remain not included in the resulting
NDJSON
(e.g., some but not all records have<start_date type="Actual">March 15, 2004</start_date>
, and this is converted to{"start_date":"March 15, 2004"}
)
EUCTR: some renaming to harmonise EU and 3rd country trial fields
- new:
e83_single_site_trial
, was (EU trials):e83_the_trial_involves_single_site_in_the_member_state_concerned
- new:
e83_single_site_trial
, was (3rd country trials):e83_will_this_trial_be_conducted_ at_a_single_site_globally
- new:
e863_trial_sites_planned_in
, was (EU trials):e863_specify_the_regions_in_which_trial_sites_are_planned
- new:
e863_trial_sites_planned_in
, was (3rd country trials):e863_specify_the_countries_outside_of_the_eea_in_which_trial_sites_are_planned
- new:
e84_multiple_sites_in_member_state
, was (EU trials):e84_the_trial_involves_multiple_sites_in_the_member_state_concerned
- new:
e840_multiple_sites_globally
, was (3rd country trials):e84_will_this_trial_be_conducted_at_multiple_sites_globally
See also #26 (comment)
Bug fixes
- corrected batch iterations over CTIS trials accommodating unclear
totalSize
response - corrected translation of some fields from the browser URL to the API call for CTGOV2 (closes #32)
- corrected minimum curl version to 5.1.0 (closes #31)
- handled errors when saving EUCTR results (e.g., too long file path name, closes #30 and #28)
Improvements
Major
- No external tools required anymore (
Cygwin
,perl
,cat
,sed
,php
functionality for transforming text, XML and NDJSON replaced by Javascript usingR
packageV8
); addresses personally communicated concerns and faciliates use of packagectrdata
in more environments (e.g., #26); consequently, this might be a breaking change for analysing certain fields, see above which fields are affected.
Other
- added results summary download for CTIS
- added documents download for ISRCTN
- factored out document download function
- ensure
dbFindFields()
returns fields for EU and 3rd country trials in EUCTR (addresses #26) - changed order of importing from CTIS into database, improved speed
- better checked data downloads, and repeat them where necessary
- factored out temporary folder creation
- added using
options(ctrdata.tempdir = ...)
if set - removed
dfListExtractKey()
, long deprecated - removed
dfMergeTwoVariablesRelevel()
, long deprecated - reorganised code file layout