Skip to content

feat(rdm.migration): Configurable auto-adding secondary community - #544

Merged
kpsherva merged 5 commits into
CERNDocumentServer:masterfrom
sakshamarora1:feat/add_publications_to_cern_research
Oct 15, 2026
Merged

kpsherva merged 5 commits into
CERNDocumentServer:masterfrom
sakshamarora1:feat/add_publications_to_cern_research

Conversation

@sakshamarora1

Copy link
Copy Markdown
Contributor

Comment thread cds_migrator_kit/rdm/records/transform/config.py Outdated
Comment thread cds_migrator_kit/rdm/records/transform/transform.py Outdated
@sakshamarora1
sakshamarora1 force-pushed the feat/add_publications_to_cern_research branch from bf187da to 4eab511 Compare July 2, 2026 13:52
@sakshamarora1
sakshamarora1 force-pushed the feat/add_publications_to_cern_research branch from 4eab511 to 53f4ac6 Compare July 30, 2026 15:44
Comment thread cds_migrator_kit/rdm/records/transform/config.py Outdated
Comment thread cds_migrator_kit/rdm/records/transform/transform.py Outdated
Comment thread cds_migrator_kit/rdm/records/transform/config.py
"""
if self.restricted or record.get("access") != "public":
return False
if any(file.get("status") for file in entry.get("files", [])):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are you sure that this covers everything? I would also check if the output json in access.files has it set to restricted or not

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested all the possible scenarios in the test cases. Could you also check that and let me know if something is missing?

There is no access.files populated in this flow. Do you mean here? It only sets enabled key

record_json_output = {
"files": self._files(record_dump),
"pids": self._pids(json_data),
"metadata": self._metadata(json_data, entry),
}

def _files(self, record_dump):
"""Transform the files of a record."""
record_dump.prepare_files()
files = record_dump.files
return {"enabled": bool(files)}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for checking this. One more case comes to mind: we can set the whole collection to restricted, in this case we shouldn't add the scientific community either

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread cds_migrator_kit/rdm/records/transform/transform.py Outdated
@sakshamarora1
sakshamarora1 force-pushed the feat/add_publications_to_cern_research branch from 149196f to 7fb15ab Compare October 6, 2026 14:47
return {"owned_by": {"user": owner}}

def _should_add_scientific_community(self):
if self.record.restricted or self.record.access_status != "public":

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could potentially on the record class object have a function which will calculate this if in more reproducible way f.e. has_any_restrictions

def _should_add_scientific_community(self):
if self.record.restricted or self.record.access_status != "public":
return False
if any(file.get("status") for file in self.dojson_entry.get("files", [])):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this I would also "socialize" as access via has_any_restricted_file method of dojson_entry object's class

@sakshamarora1
sakshamarora1 force-pushed the feat/add_publications_to_cern_research branch from 7fb15ab to 0e069b1 Compare October 15, 2026 09:36

@kpsherva kpsherva left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, @zzacharo could you double check?

Comment on lines +128 to +130
self.restricted
or self.access_status != "public"
or self.has_any_restricted_file()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we have 3 things here that say restricted....... can't we use the compute_access output to say it is restricted or not? If the correct value is added in self.restricted then we dont need the other 2 checks....

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is going to be the consolidation part that we need to do with the new dump, compute access I think does not cover it

@kpsherva
kpsherva merged commit 78785cb into CERNDocumentServer:master Oct 15, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Migration workflow: add research types to CERN_SCIENTIFIC_COMMUNITY

3 participants