Traditional Craft Training and Demonstration in Museums: History
Please note this is an old version of this entry, which may differ significantly from the current revision.
Subjects: Others
Contributor:

Crafts that are transmitted as Cultural Heritage (CH) are called Traditional Crafts (TCs) and exhibit tangible and intangible dimensions. Traditional Crafts are presented in ethnographic museums and collections around the world, and have recently received the attention of the scientific community and the public. 

  • traditional crafts
  • craft representation
  • craft presentation
  • augmented reality
  • virtual humans
  • storytelling
  • 3D representation

1. Process Representation

For the representation, the approach proposed by Mingei [67] were adopted . In this approach, to encode craft understanding, activity diagrams are borrowed from the Unified Modeling Language (UML) [68], and used in the following sense. While UML represents computational actions that transform data, in this work it is physical actions that transform the materials. The transition types Transition, Fork, Merge, Join, and Branch are adopted and denoted as in UML. Activity diagrams can be defined hierarchically, allowing for the increase in representation detail at later stages. Moreover, their visual nature was found to support collaboration with practitioners. The progression of sequential steps is modeled by a Transition link. Forks are used to represent the initiation of two parallel tasks. In Merge transition, two or more control paths unite, and Join connects steps that should be completed before the transition to the next step. The Merge and Join transitions are structurally similar, but a Join is a synchronization across a set of parallel flows, while in a Merge only a single flow is active. Finally, Branch transitions connect a step with a decision step that accepts tokens on one incoming edge and selects one outgoing alternative. Branch nodes control the flow of a process by selecting one of several alternatives, based on the outcome of a condition evaluation.
To formally represent the annual process of mastic cultivation and the production of mastic chewing gum, activity diagrams were created and then encoded with the process schema in the MOP.
The annual process is carried out by cultivators, including the typical cultivation activities such as pruning, cleaning, fertilization, and irrigation. Cleaning the soil is an important activity that takes place from the end of June until the beginning of July, where producers clean, level, and press the soil under the trees. Then, the soil is covered with white clay so that the mastic resin that falls will stay clean. Embroidering takes place in July and August. Once dry, the collection of the dried mastic resin takes place from mid-August till mid-October. Usually, big mastic pieces of resin fall on the soil, while mastic “tears” remain and dry on the bark and the branches. Sifting helps to separate the mastic gum from gathered dirt and leaves. After sifting, the producers clean the gum with soap and plenty of water. The producers make a first classification of the mastic gum that is collected. After the classification, the mastic gum is sent to the Chios Mastic Growers Association in November. The activity diagram is shown in Figure 9.
Figure 9. UML diagram of the mastic cultivation process.
For the production of mastic chewing gum, first, the mixture is prepared using mastic, sugar, butter, corn flour, and water. The ingredients are placed in the blending machine to produce the mixture. After 15 min, the mixture is taken out of the blending machine and is placed on a marble counter. Then, it is formed into pieces with a maximum height of 3 cm and left to cool. After cooling, the pieces are transferred to the press and engraving machine, where they are pressed and gum pieces are formed. In the end, the pieces are cut and put in the candy machine to create a coating made out of syrup. The activity diagram is shown in Figure 10.
Figure 10. UML diagram of the production of mastic chewing gum.
The activity diagrams are transcribed into transition graphs using the MOP UI. Data fields are used to enter appellations, informal descriptions, and the step order. Transitions are instantiated via dynamic UI components that adapt to transition type. An example of a represented process schema in the MOP for mastic chewing gum production is presented in Figure 11.
Figure 11. Process schema representations in MOP.

Representation of Narratives

In this work, a narrative is to be presented by a narrator in a form provided by a narration and in a format identified as a presentation. An example is a VH (narrator) presenting orally (an instance of a presentation) a narration (an instance of a narration that encodes a narrative presented in the museum). These entities are encoded with the help of the MOP. The narrative authoring component allows a heritage professional (i.e., a data curator, craft researcher, etc.) to synthesize a narrative using semantically linked data (i.e., basic data entries, events, etc.). Furthermore, recognizing the diversity of target audiences and technologies used in the project, this component supports the creation of different narration styles. Different presentations are intended for different presentation media as shown in the following section.

2. Craft Presentation and Preservation

Craft presentations built on top of the aforementioned semantic representation are delivered through multiple dissemination channels. Among these, in this section, we present (a) the digital preservation of knowledge and dissemination in open data formats, (b) web access to craft knowledge and narratives, (c) craft demonstrations via 3D visualizations, (d) virtual narrations, and (e) immersive context presentations.

2.1. Digital Preservation and Open Data Dissemination

The digital assets hosted in the MOP repository are provided online in conventional and open formats. Each asset has a unique Internationalized Resource Identifier (IRI) to be directly integrated by third parties. Our knowledge is available on the Semantic Web via the SPARQL endpoint by the MOP. Furthermore, to ensure compatibility with online knowledge sources, definitions of terms are imported to the MOP by linking to terms from the Getty Arts and Architecture thesaurus [69] and the UNESCO thesaurus [70]. For further exploitation of the semantic knowledge encoded in the MOP, a Europeana Data Model (EDM) [21] export facility has been also been implemented allowing (a) the export of data in semantic compatible with EDM format and (b) the formulation of SPARQL queries [71] to the MOP SPARQL endpoint to receive EDM-formatted results.

2.2. Web-Based Access to Knowledge and Narratives

The represented knowledge network is available through the WWW and MOP [63] in hypertext format. Semantic links are implemented as hyperlinks that lead to the pages of cited entities. Contents are also organized and presented thematically, per class type. Documentation pages contain links to digital assets, textual presentation of metadata, and previews of the associated digital assets. For locations and events, specific UI modules are provided. For locations, embedded, dynamic maps are provided through OpenStreetMap [72]. Timeline and calendar views are available for events. An example of narrative presentation through the MOP is presented in Figure 12.
Figure 12. A presentation of a narrative in MOP.
Process presentations are presented containing links to the recordings of the knowledge elements for the tools and materials, involving the participating practitioners, the date, the tools employed, and the location of the recording. If the process follows a process schema, a link to that schema and its preview are also provided. The hierarchy of process steps is presented using insets, each one presenting textual information and previews of the available digital assets. To present step organization, insets are dynamically unfolded to any depth in the process hierarchy, associated with image previews and embedded videos. Variations include images and textural descriptions.

2.3. Craft Presentations by Virtual Humans

2.3.1. Virtual Human Models of Craftspersons and Narrators

The demonstrations of 3D virtual humans, with their corresponding garments and accessories, have been carried out based on cultural and historical information and sources. Eight VHs were created using references from the ethnographic research. For the definition of the 3D meshes and the design of the skin surfaces, a customized [73,74,75,76] toolchain was employed that combines automatic generation methods with manual editing and refining. The approach utilizes VH creation software and 3D modeling software for clothes and accessories. The final model inherits and encapsulates the required structural components, such as bone and skin attachment data. This allows the VH to be dynamically animated and to exhibit real-time animation capabilities.
Another important aspect for attaining a high level of realism is the motion, behavior, and natural interaction of the VH with users, to avoid the “uncanny valley” effect [77,78]. The VH must have the ability for verbal as well as nonverbal communication skills, be intelligent, have natural communication with the users, perceive information from the user, and physically interact with them. The deformation of the skinned characters during their movements must be realistic, smooth, and not contain any discontinuities. The VH that supports all these capabilities will be an intuitive craft master acting effectively as an elderly companion and assistant for elder people.
Garment design was also considered during the implementation of avatars. To ensure appropriate rigging, since the generated models are automatically rigged either by Mixamo [76] for the VH that do not require facial animation, or by CC3 [74] for the VH that require facial animation (storytellers’ avatars), an additional check was performed to ensure that the rig is applied correctly, and the bones are well adjusted to the 3D body (see Figure 13).
Figure 13. (a) Checking bone positions. (b) Rigged model assuming poses.

2.3.2. Processing Craft Actions and Retargeting to VHs

The processing of the motion data was completed on Autodesk MotionBuilder software [79], which is dedicated to animation and the direct integration of motion capture technologies. The process requires different steps: (a) the creation of an “actor” in MotionBuilder with a skeleton definition corresponding to the BVH hierarchy, (b) the mapping of the received animations onto the actor, and (c) the synchronization of the VH with the actor by adjusting the two models so that the measurements match and the animations are correctly reproduced (retargeting, see Figure 14).
Figure 14. Examples of animating a VH using BVH input from MoCap recordings.
Processed animations were used for the implementation of the 3D representation of the mastic cultivation activities in a virtual environment depicting a mastic tree field, as shown in Figure 15. In this example, the processed gestural know-how of the captured craft practitioner is used not only for replicating the movements of the practitioner, but also setting digitized craft tools in motion using a technical approach for attaching tools to VHs [80], and by inferring tool motion from human motion [81].
Figure 15. Demonstration of the mastic cultivation activities by a VH.

2.3.3. Virtual Narrations

Virtual narrations were supported both in 3D and in AR. Within an AR context, exhibits of the Chios Mastic Museum can be augmented with VHs. Viewing the machines through the museum’s tablets, the visitors see VHs standing next to them, ready to share their stories and explain the functionality of the respective machines. A similar experience can be provided offline, through a virtual tour traveling back in time in the mast factory and interacting with workers (see Figure 16).
Figure 16. (a) Virtual factory with machines, (b) Examples of narrator VHs is the virtual factory.

2.4. Craft Training Employing Gestural Know-How on Mastic Cultivation

A craft training application for mastic cultivation was created using Hidden Markov Models for performing gesture recognition. For feedback, each one of the gestures was mapped to an abstract sound that was affected by the success factor of the user performing a gesture. An instance of the application is shown in Figure 17. Since the movement of the expert does not have details concerning the dexterous movement of the fingers, the video of the expert mastic cultivator has been placed on the top left of the screen, while in a bigger panel in the middle of the screen, there is the real-time recording of the video with the skeleton extracted from the OpenPose framework.
Figure 17. An instance of the main screen of the interactive installation.

2.5. Geographical Context Presentation

Information on geographical location and context shows the environmental aspects affecting craft practice and development. We developed Airborne, an immersive flight simulator allowing users to fly over various mastic villages in Chios. During the flyover, users can stop at each village and retrieve multimedia and text information related to those villages as shown in Figure 18.
Figure 18. Airborne indicative screens.

3. Exhibition

A subset of the presentation technologies presented in the previous section was used to implement an interactive exhibition at the Chios Mastic Museum. The objective was to enhance the presentation of intangible and tangible dimensions of the TC through targeted interactive presentations. The museum can be seen in Figure 19.
Figure 19. The Chios Mastic Museum.
As shown in this figure, the main areas of intervention were the main exhibition space of the museum and the rural space outside the museum. In the main space of the museum, this work targeted the mastic factory area that is less noticed by museum visitors since it is located in a space under the main exhibition. Furthermore, educational applications, such as craft training and context presentation, were installed in the multimedia space of the museum dedicated to children, school visits, and learning. In the rural space of the museum, there is a mastic field with metal sculpts that represent everyday people in the mastic field. The exterior interventions regarded the usage of AR for the presentation of craft activities in the mastic field. Each of the aforementioned interventions is presented thoroughly in the following sections.

3.1. Craft Presentations

3.1.1. Narrations in the Mastic Factory Exhibition

The installation at the mastic factory exhibition room is comprised of four tablet devices mounted on floor mounter bases that are located in four main spots of the museum. From each tablet, a specific area of the museum is covered and augmented through the camera of the tablet with hot spots, as shown in Figure 20. In each hot spot exist one or more stories to be told. By selecting the hotspot, a VH appears that is the visual twin of a persona used to working in the factory and operating one of the machines in visual approximation to its location. When the hot spot is selected, the VH appears in the factory through the camera of the table to narrate his life story and his daily life and work at the factory (see Figure 21).
Figure 20. A view of the available hot spots for a specific area of the museum.
Figure 21. A VH appears through AR on the screen showing the available narrations.

3.1.2. Craft Presentations in the Mastic Field

One of the requirements of presenting the craft was to display its seasonality. Very often, visitors do not understand the complete process of mastic cultivation. Another necessity for the museum was the exploitation of the external spaces and the beautiful mastic tree field, which is often overlooked by visitors. It also became evident that this “guided” tour (it is a simple and straight path through the phases of mastic cultivation, harvesting, and cleaning) is a perfect example of the yearlong process. Visitors to the rural space outside the museum can experience mastic cultivation in the field through their mobile devices. The application facilitates an AR-capable device to recognize metallic sculptures that exist in the rural space of the museum. Through the camera, these sculptures become alive to present typical cultivation activities. Figure 22 presents an example of app screens, with an emphasis in the middle on the AR augmentation of the sculpts that are part of the mastic field with animations of the cultivation process.
Figure 22. Mobile app navigation example.

3.2. Craft Training

Craft training is intended for demonstrating the cultivation activities to visitors of the museum, thus providing a more immersive experience. This application was installed on the ground floor of the museum in the multimedia space. The installation is comprised of a personal computer and a monitor, together with a depth sensor for tracking the user’s actions. The user stands in front of the installation and follows the instructions provided on the screen to mimic craft actions. An example of this process is presented in Figure 23.
Figure 23. Craft training through replication of the gestural know-how of the mastic cultivator.

3.3. Presentation of Geographic Context

Airborne was installed in the multimedia room of the museum (see Figure 24). The setup was very simple and straightforward, as it involved a desktop computer set. There are two options available, (a) automated tour and (b) flight simulator. The automated tour targets users that wish to explore the mastic villages in a movie-like way, while the flight simulator is more gameplay-oriented, since users have control of the virtual drone flying in the Chios sky and are free to explore information in any way they like. In the future, the setup will be updated with a large touch-enabled screen to enhance the gameplay of the installation.
Figure 24. An instance of the main screen of the interactive installation.

3.4. Preliminary Evaluation

After the installation, a small preliminary evaluation was conducted through the approach of user observation. A user experience evaluator observed the interaction of users with the various installations at the museum to record any usability issues encountered during the interaction. Identified issues were documented to produce an evaluation report to be used for the improvement of the interactive applications. In the future, this, of course, will be followed by a formal user-based evaluation where museum visitors are recruited with their consent and requested to perform certain tasks. During the interaction, observation is employed, and after completing the tasks the users will be requested to fill in a user experience questionnaire. The preliminary evaluation was targeted at fine-tuning the installations before the final user-based evaluation to ensure that basic issues were identified and resolved before the evaluation, and thus more targeted feedback could be received.

This entry is adapted from the peer-reviewed paper 10.3390/heritage5010025

This entry is offline, you can click here to edit this entry!