• css.php

    Spatializing / Sonifying Kissinger: New Perspectives in Space and Sound

    by  • May 13, 2016 • 2015-2016 Provost Digital Innovation Grant Winners

    Project Name: Spatializing / Sonifying Kissinger: New Perspectives in Space and Sound
    Grantee: Micki Kaufman
    Discipline: History
    Funding Cycle: 2015-2016
    Project Status: Cycle Complete

    Perhaps more than any other former Secretary of State or National Security Advisor, the public ‘celebrity’ of Henry Kissinger was (and remains) characterized by an array of paradoxes. As compelling to historians as his foreign policy behavior, these internal contradictions within Kissinger’s psychology and personality (and the extent to which they affected the ‘Nixinger’ foreign policy) have been the subject of much of the recent historical study of ‘Kissingerology,’ but unsurprisingly scholars who have applied traditional policy-centric interpretive models to these studies have faced significant difficulty in their efforts. A recipient of the Provost’s Digital Innovation Grant for four years running, this project leverages the digital humanities to explore alternate means of surfacing such facets of a public figure’s persona. Combined with novel forms of DH knowledge production like ‘distant/networked reading’ and ‘aesthesis,’ and made comprehensible using the languages of visual design, computer science and text analysis (word frequency analysis, topic modeling, sentiment analysis and corpus linguistics), “Quantifying Kissinger” has peered into the Kissinger/Nixon White House in new ways, suggesting new answers to existing questions in the historiography and permitting entirely new questions to be asked. In the last grant cycle, the project explored new analytical tools and methods, including a deep analysis of word collocation on a timeline, the use of David McClure’s Textplot for frequency analysis, and developing enhancements in user interactivity using d3.js-based interfaces to the Kissinger MALLET topic models using ‘stream graphs’ and ‘force-directed’ diagrams. The next phase of the project takes this effort to another level, exploring the use of additional spatial and temporal dimensions to ‘visualize’ sequence, alternativity, simultaneity, and other patterns using space and sound, in order to facilitate new insights into Henry Kissinger’s policy and personality.

    During the past three cycles of support under the Provost’s Digital Innovation Grant the “Quantifying Kissinger” project has explored the use of text analysis and data visualization in an ongoing effort to research patterns in the correspondence of the former US Secretary of State and National Security Advisor. Despite the great promise (and productivity) of these analyses and visualizations, there remain vast and largely untapped avenues for further exploration of the data – including those found in methods that challenge the perceived limits of space and time. The use of ‘3d spatialization’ and ‘sonification’ techniques on the Kissinger text analysis data has attempted to creatively explore these limits whilst achieving practical research results. As described on the project’s video segment titled “Rotating the Potato”, trans-dimensional analyses done to date during the project’s current “Alpha” phase have already begun to illuminate fascinating possibilities for deeper analysis of the data and innovations in methodology as well.

    Whatever depth a narrative may possess, each page of the book in which it is contained is two-dimensional, and its digital analog (a window on a computer screen) is similarly two- dimensional. Whether one uses graphs, maps or trees for visualization of textual data, the modality of most ‘computer displays’ also remains flat, occupying and exploiting only two of the three spatial dimensions (‘x’ and ‘y’ but not ‘z’) of the universe in which we inhabit. However, the relationship between the words on the page is far less planar. Indeed, much can be done within these two dimensions to communicate information as the long history of data visualization can attest. For example, when ‘false perspective’ is employed to simulate a higher dimensionality to the structure of the data, the human ability to recognize depth is leveraged and more complex information can be presented and interpreted in a single image. For example, the ‘stacked bar graph’ of topic model data shows the ‘topic model’ weighting between 80 topics and over 18,000 Kissinger memoranda and telephone transcripts in one ‘distant reading’, essentially stacking 80 line graphs. [See: Figure 0.]

    Figure 0. (2012) Early stacked Microsoft Excel 3d bar graph showing two MALLET topic models for the Kissinger memcons (top) and telcons (bottom). ‘Source time’ is left-to-right. Note how the eye and mind seeks patterns across the rows of topic model data, including the alternating density between the two graphs’ left and right halves of the timeline. The threshold where topic density shifts maps to September, 1973, indicating the changes in responsibility and focus accompanying Kissinger’s promotion to Secretary of State.

    Figure 0. (2012) Early stacked Microsoft Excel 3d bar graph showing two MALLET topic models for the Kissinger memcons (top) and telcons (bottom). ‘Source time’ is left-to-right. Note how the eye and mind seeks patterns across the rows of topic model data, including the alternating density between the two graphs’ left and right halves of the timeline. The threshold where topic density shifts maps to September, 1973, indicating the changes in responsibility and focus accompanying Kissinger’s promotion to Secretary of State.

    In practice, trans-dimensional visualization surfaces new relations and patterns within the correspondence, facilitates further close/networked reading and enhancing interpretative opportunities. Using an additional spatial dimension, even false perspective, can result in a deeply informative visualization, demonstrating the potential to leverage advanced pattern and depth recognition skills in the observer. Simply adding ‘artificial parallax’ effect to 2d images has been used to great effect in a number of fields, including astronomy, but even this engaging image is profoundly limited: the position and perspective of the observer is fixed and non-interactive. A logical next step was to create this data visualization as an object in 3d, using ‘real’ spatial data, rather than artificial parallax.

    Using Google Sketchup, the topic model data was converted into a gray scale terrain map, and the surface of a ‘mesh object’ extruded to a height matching the data value for each topic during each month of the 9-year time span. Occupying the graph and walking around within it provides a very different informational experience than the elevated, fixed view of false perspective. The graph can be rotated, and even be walked through, the interactivity and novel additional dimension combining to facilitate an unexpected, serendipitous interpretative experience for the observer. What is the ‘true’ importance of peaks and valleys, when one’s own position within the visualization is relative? A different answer emerges from within, rather than above, the graph. [See: Figure 1.]

    Figure 1. A view from –within-- the landscape of a Google Sketchup spatialization of the same MALLET topic model data. Note the parallels in the ‘peaks’ and ‘valleys’ of data from foreground to background (in this view, ‘source time’ is left->right) and the importance of the user’s ‘altitude’ in determining which peaks, valleys and other features are ‘interesting’. An animated walkthrough can be seen here (use the menu to select the ‘Sketchup’ section, or move the slider to begin at 00:54): http://quantifyingkissinger.com/rotatingthepotato/

    Figure 1. A view from –within– the landscape of a Google Sketchup spatialization of the same MALLET topic model data. Note the parallels in the ‘peaks’ and ‘valleys’ of data from foreground to background (in this view, ‘source time’ is left->right) and the importance of the user’s ‘altitude’ in determining which peaks, valleys and other features are ‘interesting’. An animated walkthrough can be seen here (use the menu to select the ‘Sketchup’ section, or move the slider to begin at 00:54): http://quantifyingkissinger.com/rotatingthepotato/

    Similarly, using Gephi’s Force Atlas 3d plugin, 3d versions of force-directed networks provide additional detail about the relationships between the topics and documents. As with Mercator projection and other geographical attempts to reconcile three-dimensional objects in two dimensions, objects placed in proximity on a flat projection of the graph may be quite ‘distant’ in terms of their relationships as expressed in the data. What is the ‘true’ distance one perceives between two points within a spherical space, when that sphere has been squashed flat, projected onto a plane? Using Gephi’s Force Atlas 3d plugin provides user interactivity and uses parallax to allow users to better differentiate the nature of these proximities – and the underlying relationships, even on a 2d computer display. [See: Figure 2.]

    Figure 2. View of a Gephi 3d force-directed network diagram based on a Textplot of the Kissinger Correspondence by David McClure. The graph can be rotated, and the clustering around ‘hub’ nodes representing prominent individuals like Scowcroft and Nixon becomes more apparent. For a live example of the usefulness of interactivity (rotation/navigation) and false perspective using parallax in 10 Gephi, please see the ‘interacting in 3d’ section of ‘Rotating the Potato,’ which begins at 07:17): http://quantifyingkissinger.com/rotatingthepotato/

    Figure 2. View of a Gephi 3d force-directed network diagram based on a Textplot of the Kissinger Correspondence by David McClure. The graph can be rotated, and the clustering around ‘hub’ nodes representing prominent individuals like Scowcroft and Nixon becomes more apparent. For a live example of the usefulness of interactivity (rotation/navigation) and false perspective using parallax in Gephi, please see the ‘interacting in 3d’ section of ‘Rotating the Potato,’ which begins at 07:17):
    http://quantifyingkissinger.com/rotatingthepotato/

    While fascinating and provocative, Google Sketchup and the Gephi 3d plugin are both somewhat rudimentary in their capabilities. The next step for exploring useful spatializations of this data is to engage a 3d artist to create richer, more interactive and intuitive visualizations, using methods and technologies with which users can interact more effectively and productively. Under the Grant, I will engage a 3d artist skilled in sophisticated 3d tools like Blender and Unity to develop and deploy them on dedicated 3d platforms like the Oculus Rift, PlayStation, Xbox and others. [See: Figure 3.]

    Figure 3. Blender prototype of a navigable 3d interface for exploring topic model data. In this view, each topic is represented along one of the 40 ribbons extending from the central hub, and the flow of ‘source time’ is pointed towards the distance. More modeling is necessary to continue to develop the idea, changing each ribbon to an ‘area graph’ of sorts whose changing distance to the center is based on the topic model weight at that time, and whose outer surface contains media (audio/video/images/source documents) representative of the documents, topics, and times in question, and which come into greater ‘focus’ as the user moves closer to that topic.

    Figure 3. Blender prototype of a navigable 3d interface for exploring topic model data. In this view, each topic is represented along one of the 40 ribbons extending from the central hub, and the flow of ‘source time’ is pointed towards the distance. More modeling is necessary to continue to develop the idea, changing each ribbon to an ‘area graph’ of sorts whose changing distance to the center is based on the topic model weight at that time, and whose outer surface contains media (audio/video/images/source documents) representative of the documents, topics, and times in question, and which come into greater ‘focus’ as the user moves closer to that topic.