1

2

3

4 

5

1 Mondrian and Rothko
Comparison between 128 paintings by Piet Mondrian (from 1904-1917 period) and 168 paintings by Mark Rothko (1934-1970).

X axis – brightness mean
Y axis – saturation mean
2 One Million Manga pages

3
Top: Three Songs About Lenin (1934)
Middle: The Eleventh Year
Bottom: A Man with The Movie Camera (1929) Film shots manually classified using 6 shot types types: normal shot, intertitle, multi Image, object animation, animated intertitle, multiple exposure. A single bar represents one shot; bar length = shot duration; bar color = shot type.
Order of shots: left to right.
Grey color = normal shot type.

4 Time Magazine covers

5 Video game traversals

Dr. Lev Manovich
Director, Software Studies Initiative, Calit2
software studies
Professor, Visual Arts Department, UCSD
visual arts
Professor, European Graduate School
e g s
Visiting Professor, De Montfort University
d m u

lab
web site
blog
image 1
image 2
image 3

Mailing address:
University of California, San Diego,
Lev Manovich, Visual Arts Department,
9500 Gilman Drive. #0084, La Jolla, CA 92093-0084, U.S.A

Lecture
March 8, 2010
London South Bank University
Dr Manovich was preceded by Professor Katherine Hayles, whom he credited as being the greatest mind in media. She discussed the subjects of deep attention, hyper attention, accelerated life, slow life and the ocean of data we now swim in. She is the author of the award-winning book: How We Became Posthuman: Virtual Bodies in Cybernetics, Literature and Informatics.

Outline of the talk by Dr Manovich, taken from software studies

1 Rise of visualization of culture
Information visualization as a key new techique for representation appropriate for information society – and also (if it’s interactive) a new technique for thinking.

2 What is visualization?

• reduction – use of graphical primitives
• layout which reveals patterns
• humanities visualization – creating layouts from actual media objects, as opposed to representing them via graphical primitives (no additional metadata is added)

Techniques:

• gather (Time montage)
• highlight (Anna Karenina, Hamlet) / continuity between a “full” media object and visualization/diagram
• sample (gameplay montages, Time slice, folder: Vertov sample)
• calculate (folder: Vertov averages)

Is this visualization?
Culture visualization today – usually visualization of metadata about artifacts and process ( see visualization of social networks at visualcomplexity.com). In contrast we want to show relations between actual artifacts .

Shall we call image graphs and other techniques which involve arranging images (or their sample) in particular ways “visualization”? If we assume that the core principle of visualization is not reduction and (therefore) use of vector primitives such as points and lines (which also form the language of diagrams and sketched in art) but rather the arrangments of elements in a layout which shows patterns/ relationships between elements, then the answer is yes.

Note also that if normal visualizations consist from symbolic signs (vector elements which stand for the objects and which signify through an agreed code), “direct visualization” uses objects themselves.

Therefore it does not involve signification or reduction (in that direct visualization parallels figurative art which also does not signify real-life objects through signs but represents them in detail, and which also communicates meaning through layout – traditionally called composition.) But while the goal of figurative art is communication of meaning, visualization only shows patterns – it’s up to the researcher to interpret them as meaningful.

20th century cultural theory and media art often focused on close reading of media – zooming in and slowing it down (think of “24 hour Psycho”) But after media explosion (social media and archives digitization) we need to learn how to zoom out, fast forward, compress (visually), summarize – so we can make sense of vast cultural landscapes.

Visualization dimensions:

• functional – aesthetic;
• using established familiar methods – inventing new methods
• Lots of artistic visualization is purely aesthetic – I.e. It’s not trying to reveal patterns but only uses visualization principle of deriving images from data to make abstract art.

Where does culturevis fit in? Ideally we want to be both functional and aesthetic (find forms which best express the particular artifacts and cultures), (examples of my different visualizations of Vertov shot lengths). However if we want to be able to compare many artifacts, we need to use the same technique.

We can use Pierce’s signs classification – icon, index, symbol, diagram – in relations to visualization. This scheme describes the types of relations between a sign and a referent. But we can also add a new dimension which describes “how much” a sign represents, so to speak – does it show more or less information about the referent, and what kind of information?

Normally, we think of visualization (i.e. a representation which uses vector elements) and a realistic image (a photograph or a painting) as opposites, one representing the bare minimum structure of an object and the other representing the object’s sensorial appearance in detail. But if we look at them using our new “how much and what is represented” dimension, we see that there are all kinds of intermediate cases. Therefore realistic representation and vector visualizations are just the extremes of a continuous dimension.

At the same time, there is a qualitative difference between Marey’s chronophotographs which diagram motion and visualizations which may represent relations which are not directly visible (such as economic data) – so on this dimension indexical diagrams (such as Marey) and visualizations are different categories.

3 Cultural analytics: data analysis + visualization
Add metadata (via manual annotation and/or automatic analysis), then visualize

Some of the key advantages of this method:

• automatic analysis + “image graphs”:

1) understanding meaning and/or cause behind the pattern (for instance, a repeating movement pattern in Vertov is sometimes due to parallel montage, but in other cases its not)

2) revealing additional patterns (for instance, changes in communication techniques across Time covers)

• visualizing (analog / continuous) cultural dimensions which can’t be adequately described with language (which uses discrete categories)
• visualizing continuously changing qualities over time

particularly useful for 21st century motion graphics and films, but also opens a new direction for understanding 20th century cinema / rhythm / time series analysis

example: movement pattern in “11th Year”
example: Time covers (change over a longer period)
example: US, French and Soviet 20th century films: comparing shot lengths

4 Beyond categories: aggregation without structure
Latour’s arguments: tracking and representing aggregates of objects, without the need to go to another level of model, structure, etc. (“Tarde’s idea of quantification” in Mattei Candea, ed. The Social After Gabriel Tarde: Debates and Assessments.)

Extending these arguments to culture visualization:
from categories (i.e. genres, historical periods, etc.) to a multi-dimensional space of features where we can see objects forming distributions and clusters.

Latour: “we should find ways to gather individual “he” and “she” without losing out on the specific ways in which they are able to mingle…. But never in some overarching society. The challenge is to try to obtain their aggregation without either shifting our attention at any point to a whole, or changing modes of inquiry.”

Similarly, we need to get away from the standard distinction betwen an individual cultural artefact and larger categories, be they “cultures,” “genres”, etc. We need new ways of studying aggregates – bottom up ( which is what data analysis + visualization make possible.) Here the ability of computers to keep tracks of large volumes of data and navigate through the data at arbitrary zoom levels without the need for aggregation, simplification or averaging becomes crucial. (Rather than seeing Manga space as a map consisting from a few distinct regions, we can show every single page and observe patterns of continuous change across this space.) Modern computing allows us to analyze, record and represent “individual variations” (Latour) of billions of entities – thus making possible for social and cultural sciences to become truly scientific in a way still inaccessible for natural sciences (because their entities – such as gas molecules – are still too numerous for computers to represent individually and therefore they have to use general models to represent their structure and behavior.

“Structure is what is imagined to fill the gaps when there is a deficit of information as to the ways any entity inherits from it’s predecessors and successors.” (We still have the problem of mapping exactly this information. However we can at least start by refusing structures such as “genre”, “period” etc.)

“Individual variations are the only phenomenon worth looking at in societies for which there are comparatively few elements.”

visualizations_vs_categories folder

• example: Manga analysis
• example: modernism folder

Latour: “through the ease with which we can navigate a datascape, we manage to interrupt the transubstantiation of the aggregate into a law, a structure, a model and complicate the way through which one monad may come to summarize the “whole.”

“But the “whole” is now nothing more than a provisional visualization which can be modified and reversed at will, by moving back to the individual cimponents, and then looking for yer other tools to regroup the sane elements into alternative assemlages.”

“The whole lost its provilleged status: we can produce out if the same data points, as many aggregates as we see fit, while reverting back at any time, to the individual components.”

Do we still need discrete categories?

(End of talk outline)

Rather than transcribe my notes, I included the outline above. However there are additional points I wish to add. He made reference also to the following items:

• Blue Brain Institute, Switzerland.
• the language of cultures
• various forms of sampling for analysis for game play, books, films and magazines, e.g. line trends, rendering whole book on one page and using colour highlights, line graphs, time slice via ImageJ – see images link, above
• functional and figurative visualization, where figurative is more aesthetic and functional can be bar charts. Ideally we would find both forms to express cultural artifacts and gradual historical changes.
• visualising of databases
• digitizing of media archives
• objective / non-objective data samples
• there are 1 billion messages sent daily on Facebook
• our analysis systems currently resemble close reading, but in order to develop we need to ‘zoom out’ (think: Google Earth.)
• The soft version of his new book: Software Takes Command is available to download from his website link above.

Reply from Dr Manovich to my e-mail:
Claire

Thanks very much for your nice comments and for adding links etc.

the key URL at the moment is culture vis

We did not add anything to shot lengths data for 1100 films but we
gave thought about adding aditional info along the lines you suggest.
Everything takes time and since my lab us ting, I had to be strategic
about how much work too put into every project. The prolem with this
sample is that it corresponds to film canon as defined by film studies
people so it’s quite biased. So I did not want to spend lots of time
working with this data set.

Best
lev

Extracts from my mail to him:
Dear Lev, just to thank you for your stellar presentation at the Southbank

I found your systems of analysis helpful to my research on more than one level.

I found it surprising that some of the audience was difficult, especially since the final comment related to the generic nature of statistic collection which is universally accepted already, and yet your methods lead to interesting and useful quick views at cultural usage of new media, which in turn of course informs future media design and planning.

There was not much time for questions at the end of your talk, but thank you for answering my question about the content of analysis in your upcoming book. In addition, I wanted to ask you with reference to your graphical data on the shot lengths of films from 1950 – 2008: have you considered adding another dimension to that data, namely the popularity of the films? Is this at all relevant to data collection and cultural analysis, or do you think that aspect is sufficiently noted via box office sales and Oscars?

I have noted various links from your online info and videos. It has given me plenty of reading and scope for research.

I am reading MA Digital Arts at University of the Arts, London, Camberwell College of Arts. Currently we are studying real time software with sensory information via electronics circuitry.

Claire Alonge BA(Hons) Design