Commit 53ce21a9 authored by Mathias Goebel's avatar Mathias Goebel 🎠
Browse files

merge result

parents 9f11799d 60650bb0
# SADE project folder
import in eXist at /db
Package for deploying template, documentation, configuration and data of the Fontane Notizbuch Project to eXist.
# build
## standard
## light
`ant fastdeploy`
This build target creates a package with the smallest subset of items to deploy in a standard installation. Insteat of a complete publication of all 67 notebooks a single one (C07, `textgrid:16b00`) is added to the list, so the deployment is much faster.
## non standard port
if the eXist-db is available via a non-standard port, the post install script needs to be changed.
# dependencies
see [package description](expath-pkg.xml) for details
- eXist shared
- SADE (Fontane edition)
- textgrid-connect (Fontane edition)
# deployment
make sure the user running eXist-db can write at `/var/lib/textgrid`.
requires valid credentials!
place a file named 'credentials.txt' with the following information in /var/lib/textgrid/fontane/
User and password used for inital publishing of all data from TextGrid, the secret is used for accessing internal area of the website.
See [post install script](post-install.xql) for details on this process.
A logfile is prepared during the deployment. Most errors are reported in `error` elements at `initlog.xml`.
During the process you can check the logfile or check if xhtml reaches 67:
"meta: " || count(xmldb:get-child-resources("/db/sade-projects/textgrid/data/xml/meta/")),
"data: " || count(xmldb:get-child-resources("/db/sade-projects/textgrid/data/xml/data/")),
"tile: " || count(xmldb:get-child-resources("/db/sade-projects/textgrid/data/xml/tile/")),
"xhtm: " || count(xmldb:get-child-collections("/db/sade-projects/textgrid/data/xml/xhtml/")),
this may result in
"meta: 222"
"data: 87"
"tile: 132
"xhtm: 67"
on complete import (but slightly changes in this numbers are ok as well, as long
as no error is reported).
<?xml version="1.0" encoding="UTF-8"?>
<project default="xar" name="textgrid-connect" basedir=".">
<property name="build.dir" value="build"/>
<property file=""/>
<property file=""/>
<target name="xar">
<copy file="expath-pkg.xml.tmpl" tofile="expath-pkg.xml" filtering="true" overwrite="true">
<filter token="project.version" value="${project.version}"/>
<mkdir dir="${build.dir}"/>
<!-- 4 some reason, ant sometimes do not override an existing file. let us force it -->
<delete file="${build.dir}/SADE-project-develop.xar"/>
<!-- create the XAR package -->
<zip basedir="." destfile="${build.dir}/SADE-project-develop.xar" excludes="expath-pkg.xml.tmpl,${build.dir},${build.dir}/**,*.zip"/>
<delete file="expath-pkg.xml"/>
<target name="fastdeploy">
<replace file="${basedir}/post-install.xql" token="$notebooks := $notebooks" value="$notebooks := 'textgrid:16b00'"/>
<antcall target="xar"/>
<!-- and undo this stuff -->
<replace file="${basedir}/post-install.xql" token="$notebooks := 'textgrid:16b00'" value="$notebooks := $notebooks"/>
<?xml version="1.0" encoding="UTF-8"?>
<package xmlns=""
<title>SADE project collection for Fontane Notizbücher</title>
<dependency package=""/>
<dependency package=""/>
<dependency package=""/>
<dependency package=""/>

8.54 KB

xquery version "3.1";
import module namespace tgconnect="" at "/db/apps/textgrid-connect/tg-connect.xql";
import module namespace tgclient="" at "/db/apps/textgrid-connect/tgclient.xqm";
declare namespace tgmd="";
declare variable $port := "8080";
declare variable $baseUrl := "http://localhost:" || $port ||"/exist/apps/SADE/textgrid/";
declare function local:test($what as xs:string) as xs:boolean {
switch ($what)
case "markdown" return local:test-do( $baseUrl || "content.html?", $testResultsExpected($what) )
case "doku-li" return local:test-do( $baseUrl || "doku.html", $testResultsExpected($what) )
case "doku-page" return local:test-do( $baseUrl || "doku.html?id=gesamtdokumentation_iii.3.10", $testResultsExpected($what) )
case "search" return local:test-do( $baseUrl || "results.html?q=luther", $testResultsExpected($what))
case "litvz" return local:test-do( $baseUrl || "literaturvz.html", $testResultsExpected($what))
default return ()
declare function local:test-do($url as xs:string, $expectation as node()) as xs:boolean {
let $response := local:test-get($url)
let $testIt :=
for $node in $response//*
(replace(serialize( $node ), "\s", "") = replace(serialize( $expectation ), "\s", ""))
if( count( $testIt[. = true()] ) gt 1 )
then true()
else false()
declare function local:test-get($url) {
httpclient:get(xs:anyURI($url), false(), ())
declare variable $testResultsExpected :=
<div class="block-header">
<h2><span class="title">INSTITUTIONEN</span><span class="decoration"></span><span class="decoration"></span><span class="decoration"></span></h2>
<li>Georg-August-Universität Göttingen, Seminar für Deutsche Philologie, <a href="">Theodor Fontane-Arbeitsstelle</a></li>
<li>Georg-August-Universität Göttingen, <a href="">Niedersächsische Staats- und Universitätsbibliothek Göttingen</a></li>
<li>Digital Research Infrastructure for the Arts and Humanities (<a href="">DARIAH-DE</a>) </li>
<li>TextGrid – Virtuelle Forschungsumgebung für die Geisteswissenschaften (<a href="">TextGrid</a>)</li>
<h3>Assoziierte Partnerin und Eigentümerin der Notizbücher Theodor Fontanes</h3>
<li><a href="">Staatsbibliothek zu Berlin</a>, Preußischer Kulturbesitz, Handschriftenabteilung </li>
map:entry("doku-li", <li>
<div> <a href="?id=gesamtdokumentation_iii.3.16">3.16 Verlinkung von Notizbuchaufzeichnungen</a>
map:entry("doku-page", <p>Fettflecken auf den Notizbuchseiten bzw. auf den auf-/angeklebten Blättern und Drucken werden weder für die Transkription noch für den Edierten Text beschrieben, codiert oder visualisiert, es sei denn, sie sind der Grund für eine unentzifferte Zeichenfolge.</p>),
map:entry("search", <span class="hi">Luther</span>),
map:entry("litvz", <ul class="nav nav-tabs" style="font-size: smaller;">
<li class="">
<a data-toggle="tab" href="#I">Fontanes Werke</a>
<li class="">
<a data-toggle="tab" href="#II">Fontanes Quellen</a>
<li class="">
<a data-toggle="tab" href="#III">Allgemeine Nachschlagewerke</a>
<li class="">
<a data-toggle="tab" href="#IV">Nachschlagewerke der Fontane-Forschung</a>
<li class="">
<a data-toggle="tab" href="#V">Forschungsliteratur</a>
<li class="">
<a data-toggle="tab" href="#VI">Internetquellen</a>
declare function local:getColor($pos, $count) {
let $steps := (510 div $count) => floor()
let $this := $pos * $steps
if( $this lt 255 )
then "255;" || $pos * $steps || ";0"
else 510 - $pos * $steps || ";255;0"
(: using static pathes since $target and $dir serve some errors in eXist-db v3.1.1 :)
declare variable $moveProject := (
xmldb:move("/db/apps/fontane/sade-projects", "/db"),
xmldb:move("/db/apps/fontane/system/config/db/sade-projects", "/db/system/config/db"),
(: xmldb:move("/db/apps/fontane/system/config/db", "/db/system/config/db", "collection.xconf"), :)
xmldb:move("/db/apps/fontane/system/autostart", "/db/system")
declare variable $configdoc := doc( "/db/sade-projects/textgrid/config.xml" );
declare variable $credentials :=
=> tokenize("\n");
declare variable $credentialInjection :=
for $line at $pos in $credentials
switch ($pos)
case 1 return update replace $configdoc//param[@key="textgrid.user"]/text() with text {$line}
case 2 return update replace $configdoc//param[@key="textgrid.password"]/text() with text {$line}
case 3 return update replace $configdoc//param[@key="secret"]/text() with text {$line}
case 4 return update replace $configdoc//param[@key="dokuwiki.user"]/text() with text {$line}
case 5 return update replace $configdoc//param[@key="dokuwiki.password"]/text() with text {$line}
default return false();
declare variable $config := map:new(for $param in $configdoc/config/param
return map:entry(string($param/@key), string($param))
declare variable $sid :=
let $log := util:log-system-out( "&#27;[30;43m initiating post-install.xql &#27;[0m" )
(: the uris we need to publish in an intended order! :)
let $filters := <filters>
<filter key="format" value="text/xml"/>
<filter key="" value="TGPR-9b18459d-3a6b-004f-b6aa-4fba2b9e1d3e"/>
(: im Titel nicht "Notizbuch" :)
let $log := util:log-system-out( "&#27;[30;43m getting non-notebook XML URIs &#27;[0m" )
let $query := "%21%28title%3A%22Notizbuch%22%29"
let $non-notebook-xmls := tgclient:tgsearch-query-filter($filters, $query, $sid, 2000, 0)
//tgmd:textgridUri/substring-before(., ".")
=> distinct-values()
let $filters := <filters>
<filter key="format" value="text/linkeditorlinkedfile"/>
<filter key="" value="TGPR-9b18459d-3a6b-004f-b6aa-4fba2b9e1d3e"/>
let $query := ""
let $log := util:log-system-out( "&#27;[30;43m getting TILE object URIs &#27;[0m" )
let $tble-objects := tgclient:tgsearch-query-filter($filters, $query, $sid, 2000, 0)
//tgmd:textgridUri/substring-before(., ".")
=> distinct-values()
let $filters := <filters>
<filter key="format" value="text/xml"/>
<filter key="" value="TGPR-9b18459d-3a6b-004f-b6aa-4fba2b9e1d3e"/>
(: im Titel "Notizbuch" :)
let $query := "%28title%3A%22Notizbuch%22%29"
let $log := util:log-system-out( "&#27;[30;43m getting notebook XML URIs &#27;[0m" )
let $notebooks := tgclient:tgsearch-query-filter($filters, $query, $sid, 2000, 0)
//tgmd:textgridUri/substring-before(., ".")
=> distinct-values()
let $notebooks := $notebooks
let $special-files := ( "textgrid:342bm" (: synonyms.txt :)
, "textgrid:34rhf" (: charmap.txt :)
let $installSequence := ( $special-files, $tble-objects, $non-notebook-xmls, $notebooks )
let $report := for $uri at $pos in $installSequence
update insert (<publish uri="{$uri}"/>) into //initlog/install/todo
let $log := if(count($notebooks) = 1)
then util:log-system-out( "&#27;[38;5;210m&#27;[48;5;33m we are on fastdeploy! &#27;[0m" )
else util:log-system-out( "&#27;[38;5;231m&#27;[48;5;52m complete import… got coffee? &#27;[0m" )
let $count := count( $installSequence )
(: there are some documents with an unknown status in the pipeline: :)
let $removeSomeObjects := ("textgrid:27656", "textgrid:27657", "textgrid:27658", "textgrid:2765b", "textgrid:2765c")
util:log-system-out( "getting DATA from TextGrid…" ),
for $uri at $pos in $installSequence[. != $removeSomeObjects]
where starts-with($uri, "textgrid:")
let $color := local:getColor($pos, $count)
let $systemOut := util:log-system-out( "&#27;[48;2;"|| $color ||"m&#27;[38;2;0;0;0m " || $uri || " &#27;[0m" )
let $response :=
try {
tgconnect:publish( $uri,
} catch * { <error code="{$err:code}" timestamp="{current-dateTime()}">{$err:description}</error> }
update insert ( <report uri="{$uri}">{$response}</report>) into //initlog/install/done,
(: INDEX :)
util:log-system-out( "reindexing…" ),
let $doc := doc( "/db/system/config/db/sade-projects/textgrid/data/xml/data/collection.xconf" )/*
let $tmp := xmldb:store( "/db", "tmp-collection.xconf", $doc )
let $log := util:log-system-out( "storing temp:" || $tmp )
let $newdoc := doc( $tmp )
let $remove := xmldb:remove( "/db/system/config/db/sade-projects/textgrid/data/xml/data", "collection.xconf"),
$log := util:log-system-out( "removing original…" )
let $store := xmldb:store( "/db/system/config/db/sade-projects/textgrid/data/xml/data", "collection.xconf", $newdoc ),
$log := util:log-system-out( "storing new:" || $store )
xmldb:remove( "/db", "tmp-collection.xconf" )
Tests will fail on autodeploy, because they are done via httprequest on localhost.
util:log-system-out( "testing…" ),
for $what in
( "markdown",
let $test := local:test($what)
update insert ( <test what="{$what}">{$test}</test> ) into //initlog/tests,
(: set correct mode for autostart trigger :)
sm:chmod(xs:anyURI("/db/system/autostart"), "rwxrwx---"),
sm:chmod(xs:anyURI("/db/system/autostart/reindex.xq"), "rwxrwx---")
xquery version "3.1";
import module namespace tgconnect="" at "/db/apps/textgrid-connect/tg-connect.xql";
declare variable $sid external;
declare variable $uri external;
declare variable $pos external;
declare variable $count external;
declare function local:getColor($pos, $count) {
let $steps := floor( 510 div $count)
let $this := $pos * $steps
if( $this lt 255 )
then "255;" || $this || ";0"
else 510 - $this || ";255;0"
let $color := local:getColor($pos, $count)
let $systemOut := util:log-system-out( "&#27;[48;2;"|| $color ||"m&#27;[38;2;0;0;0m " || $uri || " &#27;[0m" )
tgconnect:publish( $uri,
<?xml version="1.0" encoding="UTF-8"?>
<meta xmlns="">
<description>SADE project folder for Fontane Notizbücher</description>
<author>Ubbo Veentjer</author>
<author>Mathias Göbel</author>
......@@ -4,25 +4,29 @@
<param key="project-title"/><!-- reference to a template to be used for given project
change the template for the project by changing the param template
TODO: how to distinguish between (sade-)global and (project-)local templates ? -->
<param key="template">tmpl1</param><!-- data-dir is relative to project-dir
<param key="template">/templates/tmpl1</param>
<!-- data-dir is relative to project-dir
default: data -->
<param key="textgrid.tgcrud"></param>
<param key="textgrid.webauth"></param>
<param key="textgrid.authZinstance"></param>
<param key="textgrid.user">mgoebel_backup</param>
<param key="textgrid.password"></param>
<param key="textgrid.user">TODO</param>
<param key="textgrid.password">TODO</param>
<param key="textgrid.public-triplestore"></param>
<param key="textgrid.nonpublic-triplestore"></param>
<param key="data-dir">data</param>
<param key="sade.user">admin</param>
<param key="sade.password"></param>
<param key="secret"></param>
<param key="sade.password">TODO</param>
<param key="secret">TODO</param>
<param key="textgrid.sidcachepath">/sade-projects/textgrid/</param>
<param key="textgrid.sidcachepath.user">admin</param>
<param key="textgrid.sidcachepath.password"></param><!-- parameters for MODULES -->
<param key="textgrid.sidcachepath.password"/><!-- parameters for MODULES -->
<param key="textgrid.digilib"></param>
<param key="textgrid.digilib.tile">dw=100&amp;dh=100</param>
<param key="textgrid.sidcachepath">/sade-projects/textgrid/</param>
<param key="sade.develop">false</param>
<param key="sade.develop">true</param>
<param key="dokuwiki.url"></param>
<param key="dokuwiki.user">TODO</param>
<param key="dokuwiki.password">TODO</param>
<module key="tmpl-nav">
<param key="location">navigation.xml</param>
......@@ -36,37 +40,38 @@
<module key="faceted-search">
<param key="viewer-html">index.html?id=/xml/data/</param><!-- how much kwic hits to show per document -->
<param key="kwic-hits">3</param>
<param key="viewer-html">edition.html?id=/xml/data/</param><!-- how much kwic hits to show per document -->
<param key="kwic-hits">20</param>
<param key="kwic-width">100</param>
<param key="thumbnail">false</param>
<param key="hits-per-page">10</param>
<param key="result-xslt">/db/sade-projects/textgrid/xslt/search-hit.xslt</param>
<param key="query-root">
<param key="result-title">
<param key="thumbnail">false</param>
<param key="facets">
<facet key="authors" title="tei:author">
<facet key="keywords" title="Schlagwort">
<facet key="dates" title="tei:date/@when">
<facet key="persons" title="tei:person">
<facet key="places" title="tei:place">
<facet key="events" title="tei:event">
<!-- see SADE/modules/faceted-search/config.xml, seems that the config is taken from there on local instances -->
<facet key="hsscript" title="Duktus">
<xpath>tei:handShift[not(contains(., "Lat"))]/@script</xpath>
<facet key="org" title="Organisation">
<xpath>tei:rs/@ref[starts-with(., 'org')]</xpath>
<facet key="plc" title="Ort">
<xpath>tei:rs/@ref[starts-with(., 'plc')]</xpath>
<facet key="psn" title="Person">
<xpath>tei:rs/@ref[starts-with(., 'psn')]</xpath>
<facet key="illu" title="Skizzen">
<facet key="wrk" title="Werk">
<xpath>tei:rs/@ref[starts-with(., 'wrk')]</xpath>
# Emma TITEL
## Jane Austen (Winchester) AUTOREN
# Abstract
Emma (Austen 1815) QUELLENANGABE , by Jane Austen is a novel about youthful hubris and the perils of misconstrued romance. As in her other novels, Austen explores the concerns and difficulties of genteel women living in Georgian-Regency England; she also creates a lively comedy of manners among her characters. [^jeweilsNeueIdFürFußnote] FOOTNOTE
# 1 Chapter One HEADING 1
## 1.1 Emma Woodhouse HEADING 2
Emma Woodhouse, handsome, clever, and rich, with a comfortable home and
happy disposition, seemed to unite some of the best blessings of existence; and
had lived nearly twenty-one years in the world with very little to distress or vex
1. She was the youngest of the two daughters of a most affectionate, indulgent
2. And had, in consequence of her sister's marriage, been mistress of his
house from a very early period.
3. Her mother had died too long ago for her to have more than an indistinct
remembrance of her caresses, and her place had been supplied by an excellent
woman as governess, who had fallen little short of a mother in
! [Steventon Parsonage.] (pfad/Bild.png) BILDUNTERSCHRIFT
Sixteen years had Miss Taylor been in Mr. Woodhouse's family, less as a governess
than a friend, very fond of both daughters, but particularly of Emma. Between
them it was more the intimacy of sisters.
> Even before Miss Taylor had ceased to hold the nominal office of governess, the
mildness of her temper had hardly allowed her to impose any restraint; and the
shadow of authority being now long passed away, they had been living together
as friend and friend very mutually attached, and Emma doing just what she
liked; highly esteeming Miss Taylor's judgment, but directed chiefly by her own.
### 1.2.3 How was she to bear the change? HEADING 3
How was she to bear the change?—It was true that her friend was going only half
a mile from them;
+ but Emma was aware that great must be the difference between a Mrs.
Weston, only half a mile from them, and a Miss Taylor in the house; (LIST AND BULLET POINTS)
+ and with all her advantages, natural and domestic, she was now in great
danger of suffering from intellectual solitude.
She dearly loved her father, but he was no companion for her. He could not meet
her in conversation, rational or playful.
# 6 References
Austen-Leigh, James Edward. 1967. A Memoir of Jane Austen. Ed. R. W. Chapman.
Oxford: Oxford University Press. (Bibliography)
Todd, Janet. 2006. The Cambridge Introduction to Jane Austen. Cambridge University
Mazmanian, Melissa. 1999. "Reviving Emma". Persuasions Online 3/2, 12-34.
#7 List of Abbreviations
+ JA – Jane Austen
[^jeweilsNeueIdFürFußnote]: <> SELFLINKED LINKS
\ No newline at end of file
[Dieses Dokument bearbeiten](/exist/apps/eXide/index.html?open=/db/sade-projects/textgrid/data/
## generell
* was ist besser, local: oder %private ?
* lieber get-query als getQuery (xpath konvention)
* SADE sollte einen generellen restxq endpunkt haben...?
* range inddex conf: <>
## search
* index config muss da sein
* kwic treffer begrenzen
* konfigurierbare facetten
## config
* menuconfig auslagern
* Nutzername und PW (selber setzen...?)
* Adresse abseits localhost
* neues sadepublish ins p2 repo
* ws4 oder ws3? --> digilib
* objekte löschen aus sade
## menu
* format überarbeiten
* target=blank
* single items auf ebene 1
* mehr als 2 ebenen (subsubitem)
* divider
* evt. können im template definitionen für divider, etc hinterlegt werden, und dann über call ins model eingeflegt, um divider, dropdown, etc templatespzifisch bauen zu können
\ No newline at end of file
An h1 header
Paragraphs are separated by a blank line.
2nd paragraph. *Italic*, **bold**, and `monospace`. Itemized lists
look like:
* this one
* that one
* the other one
Note that --- not considering the asterisk --- the actual text
content starts at 4-columns in.
> Block quotes are
> written like so.
> They can span multiple paragraphs,
> if you like.
Use 3 dashes for an em-dash. Use 2 dashes for ranges (ex., "it's all