README.md 4.9 KB
Newer Older
Marcel Hellkamp's avatar
Marcel Hellkamp committed
1
2
3
# pycdstar3: Library and command-line client for GWDG CDSTAR 3.x

This is a client library and command-line toolbelt for accessing a
Marcel Hellkamp's avatar
Marcel Hellkamp committed
4
[GWDG CDSTAR 3](https://cdstar.gwdg.de/) repository, written in Python (3.4+) and designed to be extensible and used for building your own tools and workflows.
Marcel Hellkamp's avatar
Marcel Hellkamp committed
5

Marcel Hellkamp's avatar
Marcel Hellkamp committed
6
There is already a library called [pycdstar](https://pypi.org/project/pycdstar/) for accessing older versions of CDSTAR. The two libraries may be merged at some point, but for now, use [pycdstar](https://pypi.org/project/pycdstar/) for CDSTAR 2 and [pycdstar3](https://gitlab.gwdg.de/cdstar/pycdstar3) for CDSTAR 3.
Marcel Hellkamp's avatar
Marcel Hellkamp committed
7

8
## Library
9

10
11
The `pycdstar3.CDStar` class closely reassembles the actual CDSTAR v3 REST API, one method per API endpoint. It provides basic connection pooling, transaction management, error handling and json parsing on top of the `requests` library, but tries to stay out of your way otherwise. Use this if you already know the CDSTAR API and need maximum control.

mhellka's avatar
mhellka committed
12
```python
13
14
15
16
17
18
19
20
from pycdstar3 import CDStar
cdstar = CDStar("https://cdstar.gwdg.de/demo/v3/", auth=("USER", "PASS"))

with cdstar.begin(autocommit=True):  # Wrap everything in a transaction (optional)
    vault_name = "demo"
    archive_id = cdstar.create_archive(vault_name).id
    cdstar.put_file(vault_name, archive_id, "test.py", __file__)
    
21
    with cdstar.get_file(vault_name, archive_id, "test.py") as stream:
22
        with open("/tmp/test.py", "wb") as target:
23
            for chunk in stream.iter_content(buffsize=1024*64):
24
25
                target.write(chunk)
```
Marcel Hellkamp's avatar
Marcel Hellkamp committed
26

27
For a more fluent and high-level approach, wrap your `CDStar` instance in a `CDStarVault`. This hides the HTTP layer behind an object-oriented API and offers convenient wrappers and methods for the most common operations.
Marcel Hellkamp's avatar
Marcel Hellkamp committed
28

mhellka's avatar
mhellka committed
29
```python
30
31
from pycdstar3 import CDStar, CDStarVault
cdstar = CDStar("https://cdstar.gwdg.de/demo/v3/", auth=("USER", "PASS"))
Marcel Hellkamp's avatar
Marcel Hellkamp committed
32

33
34
with cdstar.begin(autocommit=True):  # Wrap everything in a transaction (optional)
    vault = CDStarVault(cdstar, "demo")
35
36
    file = vault.new_archive().file("test.py")
    file.put(__file__)
Marcel Hellkamp's avatar
Marcel Hellkamp committed
37

38
39
    with file.stream() as stream:
        stream.save_to("/tmp/")
40
```
Marcel Hellkamp's avatar
Marcel Hellkamp committed
41

42
43
44
## Command-Line interface

`pycdstar3` is also a command-line toolbox to upload, download or manage data in a CDSTAR repository.
45

46
Please note that the command-line interface is made for humans, not scripts. The commands and output may change between releases. If you want to automate CDSTAR, consider implementing your tools directly against the `pycdstar3` client library.
47

48
### Workspace mode
49

50
Most `pycdstar3` commands require `--server` and `--vault` arguments to be set, or `CDSTAR_SERVER` and `CDSTAR_VAULT` environment variables to be defined. As an alternative, you can run `pycdstar3 init` to create a *workspace directory*. When within this directory (or any sub-directory), the client will switch to *workspace mode* and read default values for `--server`, `--vault` and other settings from workspace configuration. This is particularly useful if you are working with multiple servers or vaults and want to easily switch between them.
51

52
53
54
```sh
# Explicit
pycdstar3 --server https://user:pass@example.com/v3/ --vault myVault search 'dcAuthor=Einstein'
Marcel Hellkamp's avatar
Marcel Hellkamp committed
55

56
57
58
59
60
61
# Environment Variables
export CDSTAR_SERVER=https://user:pass@example.com/v3/
export CDSTAR_VAULT=myVault
pycdstar3 search 'dcAuthor=Einstein'

# Workspace mode
mhellka's avatar
mhellka committed
62
pycdstar3 init  # creates .cdstar/workspace.conf
63
64
pycdstar3 search 'dcAuthor=Einstein'
```
Marcel Hellkamp's avatar
Marcel Hellkamp committed
65

Marcel Hellkamp's avatar
Marcel Hellkamp committed
66
### Command Usage
67

Marcel Hellkamp's avatar
Marcel Hellkamp committed
68
This is an (incomplete) list of (planned) commands. For a complete list, run `pycdstar3 -h` and for details, see `pycdstar3 COMMAND -h`.
69

70
* **`init`**: Create a new workspace. 
Marcel Hellkamp's avatar
Marcel Hellkamp committed
71
72
73
74
75
76
77
78
79
80
81
82
* **`new`** Create a new (empty) archive.
* **`info`** Query information about vaults, archives or files.
* **`meta get/set/list`** Manage metadata attributes for archives or files.
* **`acl get/set/list`** Manage access control entries and permissions.
* **`put`** Upload one or more files or folders to an archive.
* **`get`** Download or stream a single file from an archive.
* **`ls`** List files in an archive.
* **`rm`** Remove archives or files.
* **`zip`** Download an entire archive as zip or tar.
* **`sync`** Sync a local folder with a remote archive.
* **`scroll`** List all IDs in a vault.
* **`search`** Search in a vault.
83

Marcel Hellkamp's avatar
Marcel Hellkamp committed
84
85
## License

Marcel Hellkamp's avatar
Marcel Hellkamp committed
86
    Copyright 2019 Gesellschaft für wissenschaftliche Datenverarbeitung mbH Göttingen
Marcel Hellkamp's avatar
Marcel Hellkamp committed
87
88
89
90
91
92
93
94
95
96
97
98
99
100
    Copyright 2019 Marcel Hellkamp
    
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.
    You may obtain a copy of the License at
    
        http://www.apache.org/licenses/LICENSE-2.0
    
    Unless required by applicable law or agreed to in writing, software
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
    limitations under the License.