README.md 5.73 KB
Newer Older
Marcel Hellkamp's avatar
Marcel Hellkamp committed
1
2
3
4
5
6
7
8
9
10
11
12
13
# pycdstar3: Library and command-line client for GWDG CDSTAR 3.x

This is a client library and command-line toolbelt for accessing a
[GWDG CDSTAR 3.x](https://cdstar.gwdg.de/) repository, written in Python (3.4+)
and designed to be extensible and used for building your own tools and workflows.

There is already a library called [pycdstar](https://pypi.org/project/pycdstar/)
for accessing older versions of CDSTAR. The two libraries may be merged at some
point, but for now, use [pycdstar](https://pypi.org/project/pycdstar/) for
CDSTAR 2 and [pycdstar3](https://gitlab.gwdg.de/cdstar/pycdstar3) for CDSTAR 3
and newer.


14
15
16
17
## Command-Line interface

`cdstar3` is a command-line toolbox to upload, download or manage data in a CDSTAR repository.

Marcel Hellkamp's avatar
Marcel Hellkamp committed
18
Please note that `cdstar3` was designed to be used by humans, not scripts. The output is mostly human-friendly and may change between releases. If you want to automate CDSTAR, consider implementing your tools using the `pycdstar3` client library, or directly against the stable CDSTAR REST API.
19
20
21
22
23

### Configuration

The `cdstar3` client needs to know which server to connect to and which vault to use per default. This information (and more) is defined in a file named `cdstar.conf`. If no such file is specified via the `-c` parameter, `cdstar3` will look for a `cdstar.conf` in the current working directory, and then in any of its parent directories. As a last resort, certain system-dependent user folders are searched (e.g. `~/.config/cdstar3/` on linux). You can create a configuration file with `cdstar3 init`.

Marcel Hellkamp's avatar
Marcel Hellkamp committed
24
25
26
27
### Target Strings

A single CDSTAR server might host multiple vaults, each containing any number of archives, each containing multiple files. These can be referenced using their full resource URI (e.g. `http(s)://$server/v3/$vault/$archive/$file`) but that would be a lot of typing if you are mostly working with a single vault at a time. For this reason, a default server and vault can be configured in your `cdstar.conf` and the target strings will get a lot shorter: Archives can be references by their ID alone, and files by `$archive/$file`. If you want to specify a different vault, you can use the slight longer `/$vault/$archive` or `/$vault/$archive/$file` forms. Notice the leading `/` if you specify a vault. Even with a `cdstar.conf` present, you can still use the full URI if needed.

28
29
30
31
32
33
### Usage

This is an (incomplete) list of commands and their most important parameters. For a complete list, run `cdstar3 -h` and for details, see `cdstar3 COMMAND -h`.

* **`init`**: Ask for server address, vault, credentials and other config options and create a `cdstar.conf` file in the current directory. 

Marcel Hellkamp's avatar
Marcel Hellkamp committed
34
35
36
37
38
39
40
41
* **`put ID [path]`** Upload one or more files or folders to an archive.
* **`get ID/FILE (path)`** Download a single file. If no path is specified, it is streamed to stdout.
* **`info ID[/FILE]`** Get information about an archive or file.
* **`meta get ID[/FILE] (FIELD)`** Get metadata about an archive or file.
* **`meta set ID[/FILE] [FIELD=VALUE]`** SET metadata about an archive or file.

###########################################

42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
High-level commands work with local archive directories (one per archive). When creating a new archive or recovering an existing archive for the first time, `cdstar3` will create a hidden `.cdstar` folder within the target directory and remember the exact location (server, vault and id) of the remote archive. Do NOT delete this folder, or the correlation between your local copy and the remote archive is lost.

* **`archive DIR`**: Create a new remote archive from a local directory.
* **`recover ARCHIVE DIR`** download an existing remote archive to a local directory.
* **`sync --up/--down`**: Synchronize a local archive directory with the corresponding remote archive by upload- or downloading missing files. This command is *save* by default: Existing files are not overwritten and no files are removed.
  
  * `--update` overwrite outdated files at the target location (comparing last modified time).
  * `--force` overwrite all files that do not match, even if the target file is newer.
  * `--delete` remove files at the target location that are not present at the source.
  * `--progress` show a fancy progress bar.
  * `--dry-run` only print what would have changed, but do not actually apply any changes.
  * `-- [files]` only sync these files or folders.

Low-level commands do not require a local archive directory and operate directly on remote archives or vaults. If called from within a local archive directory however, the remote archive can be referred to as `origin`. For example, `cdstar3 ls origin` will list all files in the remote archive.   

* **`search QUERY`** Search a vault.
* **`scroll [START]`** List all IDs in a vault, starting with the given id.
* **`info [ARCHIVE]`**: Print information about a remote archive.
* **`ls ARCHIVE`** List all files in an remote archive.
* **`get ARCHIVE NAME [FILE]`** Download a single file from a remote archive.
* **`put ARCHIVE FILE [NAME]`** Upload a single file to a remote archive.
* **`meta get/set ARCHIVE ATTR [VALUES]`** Read or set the value of a meta attribute.
* **`acl ARCHIVE SUBJECT [PERMISSIONS]`** Set permissions for a specific subject.
Marcel Hellkamp's avatar
Marcel Hellkamp committed
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81

## License

    Copyright 2019 Marcel Hellkamp
    
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.
    You may obtain a copy of the License at
    
        http://www.apache.org/licenses/LICENSE-2.0
    
    Unless required by applicable law or agreed to in writing, software
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
    limitations under the License.