Digital Morphology of Rocaille Ornamentation

Methods at the Interface of Art History and Computer Vision for the Analysis, Modeling, and Retrieval of Rocaille Forms in 18th-Century Augsburg Prints

Like shell or bark, like rock or foam – hardly any European ornament is at once as varied and as distinctive as the Rocaille, the defining form of the Rococo. Emerging in France around 1730, it became the predominant ornament of the 18th century, especially in Central Europe, and spread across the continent largely through ornamental prints. Yet its irregular, continuously transforming, shell-like forms resist both geometric and iconographic description, which has left central questions of dating, attribution, and scholarly evaluation open to this day.

This project addresses that gap with an interdisciplinary approach at the interface of art history and computer science, applying methods of computer vision and machine learning to ornamental form analysis. Building on the established art-historical decomposition of the Rocaille into a skeleton of C- and S-shaped volute clasps and the polymorphic combs attached to it, we developed image-based learning methods: pixel-accurate semantic segmentation of volutes and combs, and, building on that, a geometric analysis of the parameter space (weight space) of neural networks as a path toward a formal morphology of the Rocaille. The central, publicly available result is a multimodal search engine and database, backed by an openly licensed research dataset and a method that makes the weight space of neural networks semantically and invertibly accessible.

The project (DFG Project Number 461631274) is a collaboration between the Computer Graphics and Virtual Reality Research Lab (CGVR) at the University of Bremen and the Institute of Art History at the University of Regensburg

Go to rocaille-ornament.de

Rocaille examples with volutes and combs
The Rocaille as a skeleton of C- and S-shaped volute clasps with attached, flat-to-sculptural comb-like extensions (left and middle). A morphologically ambiguous ("hard") example without volutes or combs (right).

The Search Engine & Database

Search engine architecture
Overview of the search engine. Each color denotes a component/service running in its own container, so the system scales horizontally by adding or removing instances as needed.

The central goal formulated in the proposal, a retrieval instrument that can relate any Rocaille composition (etching, painting, carving, or goldsmith's work) to possible Augsburg models, was realized as an online, multimodal search engine and database. It rests on a deliberately lean, containerized architecture: a PostgreSQL database, a uniform REST interface generated with PostgREST, and a server-rendered web frontend written in Go with HTMX. Full-text and fuzzy queries are served directly by database indices without any embedding model. In addition, a multimodal embedding model (currently a fine-tuned JinaCLIP v2) enables semantic queries that may consist of text and/or image and are processed bilingually (German/English). The entire corpus is accessible both through the website and programmatically through the same REST API.

The first stage of the system was presented at Digital Heritage 2025; it has since been extended with the embedding model to support multimodal semantic search over text and images.

The Dataset

The curated corpus was published as a standalone, openly licensed research-data publication and, with a meticulous description of curation and corpus, applied as a benchmark for several neural networks and tasks. It comprises 1,611 high-resolution digitizations from major German collections (Staats- und Stadtbibliothek Augsburg, Staatsgalerie Stuttgart, Staatliche Graphische Sammlung München), of which 229 carry pixel-accurate segmentation masks distinguishing volute clasps and combs, together with bilingual metadata and expert commentary on the macro level (iconography) and micro level (ornament morphology). To our knowledge, this combination of image, segmentation, structured metadata, and curated commentary is unique for the Rocaille.

From a machine-learning perspective, the dataset is a demanding benchmark: it is small yet domain-specific, with morphologically complex targets and substantial class imbalance, so it cannot be treated trivially by generic pre-training (e.g., on ImageNet). To demonstrate its viability, encoder–decoder architectures (among them ConvNeXt V2 and SegFormer/MiT) were pre-trained with diffusion-based self-supervised learning and fine-tuned for Rocaille segmentation, with all model weights released for full reproducibility. A general foundation model (SAM 2) performed weakest, underscoring the need for domain-specific training.

Dataset sample: original print Dataset sample: segmentation mask
Show metadata (JSON) 
{
  "aufnahmenummer": "ADD0902",
  "titel": "„Deux sortes de carnes propres aux ais de Tables.“ / „Zweyerley Arthen von Eck-Stücken zu Tisch-Blatten gehörig.“",
  "anmerkungen": "in den Ecken Bildfelder mit Amor und Minerva; Voluten hier zu einem dichten, mit Rocaille durchsetztem Gerüst verknüpft, Rocaille hier recht weich, vegetabile Anklänge; Akanthusformen",
  "anmerkungen_en": "In the corners of image fields with amor and minerva; Volute here to a dense scaffolding, which is powered with rocaille, rocaille here quite soft, vegetable hints; Acanthus shapes",
  "digitales_bild": "Staats- und Stadtbibliothek Augsburg Graph Fridrich, Jac. Andr. 37",
  "num_in_folge": {
    "folge_name": "Folge von 4 Blättern",
    "folge_name_en": "Sequence of 4 Sheets",
    "blatt": 2,
    "anzahl_blaetter": 4
  },
  "personen": [
    {
      "name": "Johann Jakob Baur",
      "role": "zeichner",
      "role_en": "artist"
    },
    {
      "name": "Jacob Andreas Fridrich",
      "role": "stecher",
      "role_en": "engraver"
    },
    {
      "name": "Johann Christian Leopold",
      "role": "verleger",
      "role_en": "publisher"
    }
  ],
  "bestand": [
    {
      "item": "Staats- und Stadtbibliothek Augsburg Graph Fridrich, Jac. Andr. 37",
      "item_en": "State and City Library Augsburg Graph Fridrich, Jac. Andr. 37"
    }
  ],
  "literatur": [
    {
      "reference": "Hermann Schmitz, Katalog der Ornamentstichsammlung der Staatlichen Kunstbibliothek Berlin, Berlin/Leipzig 1939, Nr. 968",
      "reference_en": "Hermann Schmitz, catalog of the ornamental collection of the State Art Library Berlin, Berlin/Leipzig 1939, No. 968"
    }
  ]
}
Example of the visual modalities of a dataset sample: image (left) with its pixel segmentation mask (right). The red pixels (label) of the mask correspond to the class "volutes". Text modalities, metadata, including a bilingual (German/English) description, are available as JSON for each of the 1,611 images. Original image resolution: 3072 × 2304.

Semantic Access to the Weight Space

Building on the architectures validated for segmentation, we developed a method that makes the parameter space (weight space) of neural networks semantically and geometrically accessible. Layers are decomposed into semantic units (convolutional filters, feed-forward neurons, attention channels); for each unit, the training trajectory is projected into a shared, low-dimensional, invertible embedding per layer, placing all units in a common geometric coordinate system. Across CNN-, MLP-, and Transformer-based models on classification and segmentation, each architecture exhibits a distinct geometric signature, and a unit's displacement in the embedding is indicative of its functional sensitivity. Because the embedding is invertible, points can be manipulated and reconstructed back into the corresponding weights, a central prerequisite for the intended analysis and generation of a Rocaille morphology over the geometry of the weight space.

Invertible weight trajectory decomposition
Decomposition and projection of semantic units in weight space: per-unit training trajectories are projected into a shared, low-dimensional, invertible embedding that can be manipulated and reconstructed back into weights.

Publications

Data, Code & Models

Following the FAIR principles, all research data are published openly and permanently. Dataset, source code (pre-processing, training, evaluation, including all configurations), and model weights are freely reusable under attribution for non-commercial purposes. The full corpus is additionally available programmatically at any time through the website and its REST interface.

Acknowledgements

This project was funded by the German Research Foundation (DFG) – Project Number 461631274.
Image courtesy of the following collections: Staatsgalerie Stuttgart, Graphische Sammlung; Staats- und Stadtbibliothek Augsburg, Graphische Sammlung; Staatliche Graphische Sammlung München.

License

This original work is copyright by University of Bremen and University of Regensburg.
Any software of this work is covered by the European Union Public Licence v1.2. To view a copy of this license, visit eur-lex.europa.eu.
Any other assets (model weights, documents, etc.) that are not part of the dataset are covered by the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. To view a copy of this license, visit creativecommons.org.
If you use any of the assets or software to produce a publication, then you must give credit and put a reference in your publication.
If you would like to use our software in proprietary software, you can obtain an exception from the above license (aka. dual licensing). Please contact zach at cs.uni-bremen dot de.