Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added design/img_src/export/requirement_analysis.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
11,340 changes: 11,340 additions & 0 deletions design/img_src/requirements_analysis.ai

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs/_data/publications.json
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
"image": "2025_CTU-Prag.jpg",
"preview": "2025_CTU-Prag_prev.jpg",
"keywords": [ "Shape-Changing Interfaces", "Elastic Displays", "Open Science", "Replication Crisis", "Human-Computer Interaction", "Software Frameworks", "Development Tools" ],
"doi": "https://doi.org/10.69558/2025007",
"doi": "https://doi.org/10.69558/2026007",
"abstract": "2025_CTU-Prag.html",
"lang": "en",
"type": "conference"
Expand Down
12 changes: 3 additions & 9 deletions docs/knowledge/case-studies/bim.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ title: "Case Study: Construction Progress Visualization"
| Features | - combining layered data with zoomable user interfaces<br> - mode switching through an external foot switch<br> - gesture development |
| Publication | *Müller, M., Lier, E., Groh, R. & Hannß, F.* (2020): **A Tangible Concept for Layered Map Visualizations: Supporting on-site Civil Engineering Construction Consultations using Elastic Displays**. In: C. Hansen, A. Nürnberger & B. Preim (Eds.): Mensch und Computer 2020 - Workshopband. Bonn, Germany: Gesellschaft für Informatik e. V. (GI). DOI: [10.18420/muc2020-ws121-368](https://doi.org/10.18420/muc2020-ws121-368). |

The eighth case study explored **combining zoomable maps and layered information for construction planning and consultation**. The profile describes the historical dSense implementation, following the conventions in the [case study overview](overview.md#case-study-profiles). The source does not list an associated repository.
The eighth case study explored **combining zoomable maps and layered information for construction planning and consultation**. The profile describes the historical dSense implementation, following the conventions in the [case study overview](overview.md#case-study-profiles).

## Research Questions

Expand Down Expand Up @@ -65,7 +65,7 @@ The two-hand lens used a fixed size. Early trials found that continuously changi

The study focused on construction execution, particularly progress monitoring, schedule updates, and cost control within a Building Information Modeling (BIM) context. It developed a map-based visualization and interaction concept rather than addressing the full project lifecycle.

Its layer model related construction phases to their physical depth. This suited the chosen civil-engineering scenario, but constrained transfer to projects with simultaneous work at different heights. The source describes map interaction, gesture development, and planning visualizations; the reported comparative user test specifically concerns mode switching.
Its layer model related construction phases to their physical depth. This suited the chosen civil-engineering scenario, but constrained transfer to projects with simultaneous work at different heights. The case study focues on map interaction, gesture development, planning visualizations, and mode switching.

**[⬆ back to top](#table-of-contents)**

Expand Down Expand Up @@ -132,12 +132,6 @@ This context-sensitive interpretation distinguished moving an entire interval fr

## Findings and Discussion

### Comparing Mode-Switching Techniques

The source reports a user test documented in *Lier (2019, pp. 146 ff.)*. The foot switch showed **significant advantages over Together-and-Apart** in task completion, completion time, usability, and user experience. Differences between the foot switch and the contact-count-based two-hand lens were smaller and **not statistically significant in that test**.

The excerpt provides no participant count, effect sizes, or detailed statistics. The result therefore supports a comparison of the tested techniques without establishing a general superiority of external controls or an evaluation of the entire planning workflow.

### Persistent Views and Physical State

A dedicated mode switch offered a further benefit: retaining the current view allowed users to step away from the wall and see the complete visualization. This addressed the difficulty of inspecting a large display while holding a deformation.
Expand All @@ -158,7 +152,7 @@ The coupling of construction phase and physical depth also had a clear boundary.

### Implications for Framework Development

For the forthcoming [requirements analysis](../../software/repo/requirements.md), the study provides evidence for the following areas:
For the forthcoming [requirements analysis]({{ site.baseurl }}/software/repo/requirements.html), the study provides evidence for the following areas:

- **Combining input sources:** Allow surface interaction to work alongside external controls such as a foot switch.
- **Application-specific gesture interpretation:** Expose multiple contact positions and deformation depths so clients can interpret gestures according to their current mode and target objects.
Expand Down
20 changes: 14 additions & 6 deletions docs/knowledge/case-studies/deep-zoom.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,12 @@ The implementation used **.NET and WPF** in a client-server architecture. Despit

This continued Glyphboard's separation of tracking from application logic, while applying it to a native desktop visualization client.

### Tracking Configuration and Development Tools

Framework development optimized depth-image analysis and introduced configurable parameters for creating and filtering the point cloud. These changes allowed processing to be adjusted to the tracking needs of the visualization independently of its image-rendering implementation.

A simple **touch-point simulator** provided synthetic contact input for application development without the physical display. Unlike FlexiWall's depth-image emulator, it simulated detected interaction points rather than a complete surface representation.

### Mapping Deformation to Magnification

The visualization combined two images: a preview at the application's native resolution, typically **1920 × 1080 pixels**, and a high-resolution detail image. The magnified view was centred on the detected finger position. Deformation depth determined how far the detail image was scaled down from its full resolution: maximum deformation exposed the full available detail, while shallower deformation reduced magnification.
Expand All @@ -63,7 +69,7 @@ An optional third image supplied annotations. It matched the detail image's reso

### Image Size and Memory Limits

The implementation stored pixel colour values in a byte array. The source reports a limit of **2,147,483,591 elements**, corresponding to approximately **30,893 × 17,377 pixels** for a 16:9 image with four RGBA channels. These dimensions remained below one gigapixel but provided approximately **16-fold magnification** relative to the preview, which was sufficient to demonstrate the interaction concepts.
The implementation stored pixel colour values in a byte array. Due to technological constraints, these arrays are limited to a size of **2,147,483,591 elements**, corresponding to approximately **30,893 × 17,377 pixels** for a 16:9 image with four RGBA channels. These dimensions remained below one gigapixel but provided approximately **16-fold magnification** relative to the preview, which was sufficient to demonstrate the interaction concepts.

This was a limit of the implementation described in the study. Loading complete detail images also constrained possible extensions: in that implementation, a second independent lens would load another copy of the entire detail image into memory. Effective multi-touch use and collaborative viewing with multiple lenses were not addressed.

Expand All @@ -75,7 +81,7 @@ This was a limit of the implementation described in the study. Loading complete

![Lens placement options]({{ site.baseurl }}/assets/img/kb/case-studies/deep-zoom_zoom-modes.png)

The case study considered five presentation concepts, illustrated in Figure 5-39 of the source:
The case study considered five presentation concepts:

| Presentation | Relationship between detail and context |
| ------------ | --------------------------------------- |
Expand All @@ -91,7 +97,7 @@ The implemented modes were **a lens with configurable offset and full-screen zoo

Several kinds of content tested the same interaction approach:

- **Astronomical imagery:** A high-resolution view of the Milky Way from ESA's Gaia project supported exploration of large-scale structure and local detail. The source credits ESA/Gaia/DPAC and identifies the image licence as CC BY-SA 3.0 IGO.
- **Astronomical imagery:** A high-resolution view of the Milky Way from ESA's Gaia project supported exploration of large-scale structure and local detail ([Source images](https://sci.esa.int/s/ApPJaGA)).
- **City panoramas:** A Dresden panorama assembled from photographs taken for the project provided a familiar setting for inspecting buildings and landmarks.
- **Maps and aerial imagery:** Pre-rendered Google Maps satellite images at different resolutions and historical aerial photographs of Dresden from 1953 connected a city-wide overview with individual streets and buildings.
- **Event plans:** Exhibition plans supplied as vector graphics were used for OUTPUT and Fit4Congress. The overview showed room and stand silhouettes; the detailed image added room names and event information.
Expand All @@ -118,9 +124,9 @@ For the Dresden panorama, the overlay labelled significant buildings at their im

## Findings and Discussion

### Combining Zoom and Information Layers
The following section provides exploratory observations about the interaction concepts and implementation.

The source reports demonstration feedback and implementation experience, without a participant count or quantitative evaluation. Its findings therefore provide exploratory design insights.
### Combining Zoom and Information Layers

The restoration comparison and fading annotations demonstrated two ways to enrich zoomable images with additional information. Maps could support multiple semantic overlays, while restoration imagery suggested opportunities to explain or document changes to an artwork. Developing interaction concepts that combine zoom with navigation through several information layers remained a central research question.

Expand All @@ -140,9 +146,11 @@ These proposals expanded the possible interaction vocabulary without establishin

### Implications for Framework Development

For the forthcoming [requirements analysis](../../software/repo/requirements.md), the study provides evidence for the following areas:
For the forthcoming [requirements analysis]({{ site.baseurl }}/software/repo/requirements.html), the study provides evidence for the following areas:

- **Independent tracking and application layers:** Maintain a clear communication boundary so that surface input can drive a separately implemented visualization client.
- **Configurable point-cloud processing:** Expose parameters for point-cloud creation and filtering so tracking can be tuned independently of the visualization.
- **Simulated interaction input:** Support application development through synthetic touch points without requiring the sensor and physical display.
- **Continuous parameter mappings:** Make position and deformation depth available for direct control of magnification, alongside mappings to speed such as those used in Glyphboard.
- **Configurable presentation and feedback:** Allow applications to adjust lens offset and provide overview context while accounting for occlusion and projection distortion.
- **Layered content and annotation visibility:** Support application concepts that relate magnification to alternative imagery and gradual information disclosure.
Expand Down
15 changes: 10 additions & 5 deletions docs/knowledge/case-studies/deep.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,9 +53,11 @@ The application retained the monolithic .NET/WPF architecture but substantially

During development, the sensor changed from the original Microsoft Kinect to Kinect 2. This prompted an initial hardware-independent sensor layer capable of processing depth data from different sensors.

Surface configuration generalized FlexiWall's depth-range mapping by explicitly defining the physical resting plane of the elastic surface and thresholds for minimum deformation. These parameters established the reference from which inward and outward deformation could be interpreted and small deviations excluded from interaction detection.

### Extracting and Stabilizing Pressure Points

The implementation detected local extrema in the depth image along both horizontal and vertical directions. Differences between neighbouring depth samples approximated partial derivatives, forming a vector field in which sign changes indicated candidate extrema. For performance, the depth image was downscaled to a quarter of its original resolution.
The implementation processed the point cloud derived from the depth image to detect local extrema along both horizontal and vertical directions. Differences between neighbouring depth samples approximated partial derivatives, forming a vector field in which sign changes indicated candidate extrema. For performance, the depth image was downscaled to a quarter of its original resolution.

Each frame produced a list of extrema. **Persistent touch identities were not implemented**, because the scenario did not require them. Multi-touch detection therefore exposed simultaneous pressure points without uniquely tracking each interaction over time.

Expand All @@ -70,7 +72,9 @@ Confidence filtering delayed initial detection. Without persistent tracking, lat

Calibration used more reference points to improve spatial accuracy at different deformation depths. It partially accounted for projective distortion and reduced drift between detected and displayed positions. Residual errors remained, especially under stronger deformation and away from the projection centre.

Technical limitations prevented transferring this calibration to the existing layer-rendering component. The earlier depth-image simulation was also incompatible with DEEP, exposing the need for a different approach to interaction emulation as a development tool.
Technical limitations prevented transferring this calibration to the existing layer-rendering component. In the subsequent consolidation, the extended calibration was also excluded from the main framework because its limited added value did not justify the effort required to perform it. This decision concerned general adoption of the procedure, rather than its implementation in the DEEP prototype.

The earlier depth-image simulation was also incompatible with DEEP, exposing the need for a different approach to interaction emulation as a development tool.

**[⬆ back to top](#table-of-contents)**

Expand Down Expand Up @@ -141,9 +145,9 @@ Pressing a project revealed an annotation in three successive stages: its title,

## Findings and Discussion

### Understandability and Response Time
The following section provides exploratory observations about the interaction concepts and implementation.

The source reports observations from iterative trials and user feedback, without a participant count or quantitative evaluation. These findings provide exploratory design insights.
### Understandability and Response Time

The physical response of the elements offered an accessible introduction to the interaction concept and helped communicate relationships in the data. However, controlling the strength of an element's influence through pressure generally required a brief explanation. After familiarization, users found the direct feedback understandable.

Expand All @@ -167,10 +171,11 @@ Hands could obscure tags during direct selection. Larger, indirect deformation g

### Implications for Framework Development

DEEP moved beyond FlexiWall's direct depth-image rendering by exposing pressure-point coordinates to application logic. It also provided concrete experience with sensor abstraction, configurable simulation, data handling, and calibration. For the forthcoming [requirements analysis](../../software/repo/requirements.md), the study provides evidence for the following areas:
DEEP moved beyond FlexiWall's direct depth-image rendering by exposing pressure-point coordinates to application logic. It also provided concrete experience with sensor abstraction, configurable simulation, data handling, and calibration. For the forthcoming [requirements analysis]({{ site.baseurl }}/software/repo/requirements.html), the study provides evidence for the following areas:

- **Interaction data and continuity:** Expose multiple positions and deformation depths; support tracking over time where moving contacts or gestures require it.
- **Configurable processing:** Make the balance between noise suppression, stable detection, and latency explicit.
- **Surface configuration:** Define the resting plane and minimum-deformation thresholds independently of application-specific visualizations.
- **Spatial alignment:** Account for deformation depth and off-centre interaction when mapping sensed positions to projected content.
- **Hardware abstraction and emulation:** Support interchangeable sensors and application development without depending on the physical display or a particular rendering technique.
- **Separation of responsibilities:** Keep sensing, interaction processing, simulation, data access, and visualization sufficiently independent to support different scenarios.
Expand Down
10 changes: 8 additions & 2 deletions docs/knowledge/case-studies/depthtouch.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ The central question was how deforming a physical surface could support the mani

The tabletop used commercially available translucent Lycra stretched into an elastic surface. Lycra was selected for its elasticity, durability, affordability, ready availability, and suitability for rear projection. The fabric contained no sensors: a Microsoft Kinect depth camera captured its deformation, while a projector and mirror projected content onto it from below.

![DepthTouch CXonstruction]({{ site.baseurl }}/assets/img/kb/case-studies/depth-touch_construction.png)
![DepthTouch Construction]({{ site.baseurl }}/assets/img/kb/case-studies/depth-touch_construction.png)

Calibration aligned the coordinate systems of the depth camera and projector. Three points defining two linearly independent vectors were mapped to adjust scaling, translation, and rotation within the surface plane. This procedure corrected alignment within that plane, so the camera still had to be oriented perpendicular to the fabric and the projector's image plane parallel to it.

Expand All @@ -70,10 +70,16 @@ The physical surface served as both the input medium and a source of haptic feed

## Findings and Discussion

The source account reports that users consistently highlighted the playful, exploratory character of the interaction and the fabric's natural invitation to deform it. They also emphasized that familiar physical behaviour made the interaction intuitively understandable, relating the concept to reality-based interaction.
The following section provides exploratory observations about the interaction concepts and implementation.

Users consistently highlighted the playful, exploratory character of the interaction and the fabric's natural invitation to deform it. They also emphasized that familiar physical behaviour made the interaction intuitively understandable, relating the concept to reality-based interaction.

The prototype exposed a distinction relevant to subsequent framework development: capturing surface deformation can support simultaneous interaction without identifying individual contacts, but it does not by itself reveal which actions users perform. DepthTouch demonstrated the opportunities of this approach for physical simulation while making its limits for interpreting user input apparent.

These observations raised the broader question of how an elastic surface could extend conventional multi-touch systems with an additional interaction dimension. DepthTouch established the conceptual and technical starting point for the later case studies, including [FlexiWall](flexiwall.md), which explored further uses of surface deformation.

### Implications for Framework Development

The vector-field calculation and three-point calibration were subsequently integrated into later framework iterations as reusable functions for processing surface deformation and mapping sensor coordinates to screen coordinates. The physics simulation remained an application-specific component, separate from the framework. This distinction allowed the underlying sensing and calibration functions to support scenarios beyond the original simulation.

**[⬆ back to top](#table-of-contents)**
Loading
Loading