orrisroot
a2bc9bb642
Labels came from a fixed table matched by subset, so Storage (no bits) matched any level the table lacked, and a level was named after the first entry it contained: the server's Brain/MINDS 2.0 folders (0x10000) showed as Storage and 0x10104 as CBS or PW Open. Labels are now composed from the bits. Projects (bit 16 and up) take the names the server reports to their members through users/current/, and otherwise Project<n>; restrict slot 1 is CBS and any other Restrict<n>. A level with no meaning is shown as Unknown (0x...). chacl reads keys built the same way, refuses public and storage before any request, and lists the user's projects with their keys when given an invalid level.
294 lines
6.7 KiB
Markdown
294 lines
6.7 KiB
Markdown
# mdrs-client-python
|
|
|
|
The mdrs-client-python is python library and a command-line client for up- and downloading files to and from MDRS based repository.
|
|
|
|
## Installation (Production)
|
|
|
|
It is recommended to use a virtual environment (`venv`) to keep your Python environment isolated.
|
|
|
|
### Option 1: Using standard `venv` + `pip`
|
|
|
|
```shell
|
|
python3 -m venv .venv
|
|
source .venv/bin/activate # On Windows: .venv\Scripts\activate
|
|
pip install .
|
|
```
|
|
|
|
### Option 2: Using `uv`
|
|
|
|
```shell
|
|
uv venv
|
|
source .venv/bin/activate # On Windows: .venv\Scripts\activate
|
|
uv pip install .
|
|
```
|
|
|
|
## Development Setup
|
|
|
|
To set up a local development environment with development tools (testing, formatting, linting):
|
|
|
|
### Option 1: Using standard `venv` + `pip`
|
|
|
|
```shell
|
|
python3 -m venv .venv
|
|
source .venv/bin/activate # On Windows: .venv\Scripts\activate
|
|
pip install -e ".[dev]"
|
|
```
|
|
|
|
### Option 2: Using `uv` (Recommended for developers)
|
|
|
|
```shell
|
|
uv sync
|
|
```
|
|
|
|
## Configuration
|
|
|
|
The following environment variables can be set, either in the shell or via a `.env` file placed in the working directory.
|
|
|
|
| Variable | Default | Description |
|
|
|---|---|---|
|
|
| `MDRS_CLIENT_CONFIG_DIRNAME` | `~/.mdrs-client` | Directory where config and login cache files are stored |
|
|
| `MDRS_CLIENT_CONCURRENT` | `10` | Number of concurrent file transfers for upload and download |
|
|
|
|
## CLI Usage
|
|
|
|
### config create
|
|
|
|
Create remote host configuration
|
|
|
|
```shell
|
|
mdrs config create neurodata https://neurodata.riken.jp/api
|
|
```
|
|
|
|
### config update
|
|
|
|
Update the URL of a registered remote host.
|
|
|
|
```shell
|
|
mdrs config update neurodata https://neurodata.riken.jp/api
|
|
```
|
|
|
|
### config list
|
|
|
|
List registered remote hosts.
|
|
|
|
```shell
|
|
mdrs config list
|
|
mdrs config ls
|
|
```
|
|
|
|
### config delete
|
|
|
|
Remove a registered remote host.
|
|
|
|
```shell
|
|
mdrs config delete neurodata
|
|
mdrs config rm neurodata
|
|
```
|
|
|
|
### login
|
|
|
|
Login to remote host
|
|
|
|
```shell
|
|
mdrs login neurodata:
|
|
Username: (enter your login name)
|
|
Password: (enter your password)
|
|
|
|
mdrs login -u USERNAME -p PASSWORD neurodata:
|
|
```
|
|
|
|
### logout
|
|
|
|
Logout from remote host
|
|
|
|
```shell
|
|
mdrs logout neurodata:
|
|
```
|
|
|
|
### whoami
|
|
|
|
Print current user name
|
|
|
|
```shell
|
|
mdrs whoami neurodata:
|
|
```
|
|
|
|
### labs
|
|
|
|
List all laboratories
|
|
|
|
```shell
|
|
mdrs labs neurodata:
|
|
```
|
|
|
|
### ls
|
|
|
|
List the folder contents. You can also specify a DOI path in the form `remote:10.xxxx/yyy.ID[/optional/subpath]`.
|
|
|
|
```shell
|
|
mdrs ls neurodata:/NIU/Repository/
|
|
mdrs ls -p SHARING_PASSWORD neurodata:/NIU/Repository/PW_Open/
|
|
mdrs ls -r neurodata:/NIU/Repository/Dataset1/
|
|
mdrs ls -J -r neurodata:/NIU/Repository/Dataset1/
|
|
|
|
# DOI access examples:
|
|
mdrs ls neurodata:10.60178/cbs.20260429-001
|
|
mdrs ls "neurodata:10.60178/cbs.20260429-001/Figure 1"
|
|
```
|
|
|
|
### mkdir
|
|
|
|
Create a new folder
|
|
|
|
```shell
|
|
mdrs mkdir neurodata:/NIU/Repository/TEST
|
|
```
|
|
|
|
### upload
|
|
|
|
Upload the file or directory
|
|
|
|
```shell
|
|
mdrs upload ./sample.dat neurodata:/NIU/Repository/TEST/
|
|
mdrs upload -r ./dataset neurodata:/NIU/Repository/TEST/
|
|
mdrs upload -r --skip-if-exists ./dataset neurodata:/NIU/Repository/TEST/
|
|
```
|
|
|
|
### download
|
|
|
|
Download the file or folder. You can also specify a DOI path.
|
|
|
|
```shell
|
|
mdrs download neurodata:/NIU/Repository/TEST/sample.dat ./
|
|
mdrs download -r neurodata:/NIU/Repository/TEST/dataset/ ./
|
|
mdrs download -p SHARING_PASSWORD neurodata:/NIU/Repository/PW_Open/Readme.dat ./
|
|
mdrs download -r --exclude /NIU/Repository/TEST/dataset/skip neurodata:/NIU/Repository/TEST/dataset/ ./
|
|
mdrs download -r --skip-if-exists neurodata:/NIU/Repository/TEST/dataset/ ./
|
|
|
|
# DOI access examples:
|
|
mdrs download neurodata:10.60178/cbs.20260429-001/README_NeuroData.md ./
|
|
mdrs download -r "neurodata:10.60178/cbs.20260429-001/Figure 1" ./
|
|
```
|
|
|
|
### mv
|
|
|
|
Move or rename the file or folder
|
|
|
|
```shell
|
|
mdrs mv neurodata:/NIU/Repository/TEST/sample.dat neurodata:/NIU/Repository/TEST2/sample2.dat
|
|
mdrs mv neurodata:/NIU/Repository/TEST/dataset neurodata:/NIU/Repository/TEST2/
|
|
```
|
|
|
|
### cp
|
|
|
|
Copy the file and folder
|
|
|
|
```shell
|
|
mdrs cp neurodata:/NIU/Repository/TEST/sample.dat neurodata:/NIU/Repository/TEST2/sample2.dat
|
|
mdrs cp -r neurodata:/NIU/Repository/TEST/dataset neurodata:/NIU/Repository/TEST2/
|
|
```
|
|
|
|
### rm
|
|
|
|
Remove the file or folder
|
|
|
|
```shell
|
|
mdrs rm neurodata:/NIU/Repository/TEST/sample.dat
|
|
mdrs rm -r neurodata:/NIU/Repository/TEST/dataset
|
|
```
|
|
|
|
### chacl
|
|
|
|
Change the folder access level. The level is `private`, `pw_open`, `cbs_open`, `project<n>_open`, or several open levels joined by `_or_`. `<n>` is the project's bit position; an invalid level lists the projects you belong to with their keys.
|
|
|
|
```shell
|
|
mdrs chacl private neurodata:/NIU/Repository/Private
|
|
mdrs chacl cbs_open -r neurodata:/NIU/Repository/CBS_Open
|
|
mdrs chacl pw_open -r -p SHARING_PASSWORD neurodata:/NIU/Repository/PW_Open
|
|
mdrs chacl cbs_or_pw_open -p SHARING_PASSWORD neurodata:/NIU/Repository/CBS_or_PW_Open
|
|
mdrs chacl project0_open neurodata:/NIU/Repository/Project_Open
|
|
```
|
|
|
|
### metadata
|
|
|
|
Get a folder metadata. You can also specify a DOI path.
|
|
|
|
```shell
|
|
mdrs metadata neurodata:/NIU/Repository/TEST/
|
|
mdrs metadata -p SHARING_PASSWORD neurodata:/NIU/Repository/PW_Open/
|
|
|
|
# DOI access examples:
|
|
mdrs metadata neurodata:10.60178/cbs.20260429-001
|
|
```
|
|
|
|
### file-metadata
|
|
|
|
Get the file metadata. You can also specify a DOI path.
|
|
|
|
```shell
|
|
mdrs file-metadata neurodata:/NIU/Repository/TEST/dataset/sample.dat
|
|
mdrs file-metadata -p SHARING_PASSWORD neurodata:/NIU/Repository/PW_Open/Readme.txt
|
|
|
|
# DOI access examples:
|
|
mdrs file-metadata "neurodata:10.60178/cbs.20260429-001/Figure 1/Figure1v3.pdf"
|
|
```
|
|
|
|
### version
|
|
|
|
Show the tool name and version number
|
|
|
|
```shell
|
|
mdrs version
|
|
```
|
|
|
|
### help
|
|
|
|
Show the help message and exit
|
|
|
|
```shell
|
|
mdrs -h
|
|
```
|
|
|
|
## Python API Usage
|
|
|
|
You can also use this package as a Python library to programmatically interact with MDRS repositories.
|
|
|
|
```python
|
|
from mdrsclient.client import MdrsClient
|
|
from mdrsclient.cache import InMemoryCache
|
|
from mdrsclient.config import InMemoryConfig
|
|
|
|
# 1. Setup in-memory configuration and cache to avoid local state files (e.g., config.ini, cache/*.json)
|
|
config = InMemoryConfig("neurodata")
|
|
config.url = "https://neurodata.riken.jp/api"
|
|
|
|
cache = InMemoryCache()
|
|
|
|
# 2. Initialize client with custom configuration and cache
|
|
client = MdrsClient.from_remote("neurodata", cache=cache, config=config)
|
|
|
|
# 3. Login to the remote server
|
|
client.login("username", "password")
|
|
|
|
# 4. Use service methods
|
|
labs = client.get_laboratories()
|
|
metadata = client.metadata("neurodata:/NIU/Repository/")
|
|
|
|
# Transfer files programmatically
|
|
client.upload("/path/to/local/data", "neurodata:/NIU/Repository/TEST/", is_recursive=True)
|
|
client.download("neurodata:/NIU/Repository/TEST/data", "/path/to/local", is_recursive=True)
|
|
```
|
|
|
|
## Testing
|
|
|
|
You can run the unit test suite using the standard library `unittest` discover runner:
|
|
|
|
```shell
|
|
.venv/bin/python -m unittest discover tests
|
|
```
|
|
|
|
## Changelog
|
|
|
|
See [CHANGELOG.md](./CHANGELOG.md) for the full change history.
|
|
|