A New GitHub Data-Set

Written by Adam Franklin-Lyons

In earlier articles on this blogroll, we have written a couple of times about extracting, analyzing and organizing medieval itineraries as a source of data for doing geographic studies of medieval movement and travel (see: “Itineraries, Gazetteers, and Roads” and “Notes on the Margins“). Currently we have compiled and organized an increasing number of itinerary data-sets including digitizing older itineraries compiled in the 19th century or early 20th century. These data-sets include multiple royal itineraries from the Crown of Aragon and dozens of episcopal itineraries from England. We are planning to expand this project to include new travel itineraries from new places around Europe. To facilitate this expansion, we have moved a portion of our data onto GitHub – a site that specializes in version control usually used for software development.

The GitHub site currently hosts several of our itineraries along with a small amount of code (written in python with the Pandas library) that allows for the conversion of itineraries to trip sets, compilation of itineraries, looking up of existing names in other itineraries or in geonames.org, and other transformations that will assist in data collection and visualization. This should help us better organize all of the diverse data points, especially by linking each location (when possible) with a geonames id, a broadly cross-referenced and computer legible reference point. This leaves the door open to connecting our projects to linked open data sometime in the future (Wikidata is probably the most famous version of a linked open data project). There are also instructions available for how to insure that the geonames id is correct, not to mention what to do when there is no obviously available geonames referent.

The medium term goal is to compile a larger bibliography along with a set of usable assignments that other scholars and teachers could use as samples in digital humanities courses or as a digital example in an appropriate history course. These assignments will provide a two-fold benefit. First, each assignment undertaken will add to the scope of the overall data-set. The bibliography will provide other opportunities for history students to create their own data which we can then add to the collection. Second, because of the scope of the data already present, students will be able to more quickly use digital tools on a larger data-set to see the potential of geographic and statistical analyses. Eventually, we will add some of the more successful visualizations along with instructions to try with new data or to modify and expand on.

It is a key difficulty in teaching digital humanities that data collection on a scope large enough to produce compelling results must often be balanced with the class time needed to actually learn the digital tools used to run the analyses (GIS tools, stats packages like R, and other tools all involve whole courses in their own right.) By creating an iterative platform with model analyses and large amounts of already usable data, students and professors can participate in each stage of the project only for as long as they have available for their course without sacrificing the ability of students to practice all the steps along with way and produce satisfying historical results. Ideally, the project components on GitHub will allow courses to contribute incomplete data that other groups can pick up and continue. This makes concrete how we have attempted to run the lab in the past – with students able to work on a project at multiple stages, passing it off to a new group of students to move further when they are done. However, the instructions, future work, and goals have generally lived in the individual heads of the professors overseeing each project rather than in more publicly usable formats.

The long-term goal is to structure the data into an SQL database for easier querying, but also to increase substantially the number of trips and itineraries available in the data-set. We are also aiming to add data from a number of letter collections, including some data-sets we have already worked on (see: “Parsing the Past“). Eventually, there should be enough data about small movements to be able to ask broader questions about European mobility in the late medieval period. If we can reach a few hundred thousand individual data points (known short trips of less than a couple days), we will be able to ask systemic questions about the nature of movement. We could look for patterns such as seasonality, the influence of topography, linguistic boundaries, or observe potential regional differences across Europe.

The extended vision of the project could resemble the Stanford Orbis project – an interactive map of the ancient world for which there is no medieval equivalent. However, while Orbis is based on original research and primary sources, the sources are more diaphanous and descriptive. The map is built on algorithms that encode assumptions based on those sources, whereas a large enough data-set of individual trips would allow for algorithms that can give travel times and methods as inductive statistical guesses. These inductive guesses would be built on an extensive underlying source base of individual trips.

So, for the moment, we have tens of thousands, not quite hundreds of thousands. If you want to contribute to the bibliography or have good suggestions for primary sources that could reasonably produce an itinerary, get in touch so we can get it on the site. If you are planning on teaching a digital history course of some sort, use the data, try out the instructions, or create your own itinerary that can compliment the data already available. If you do try out some of the methods on the site, please let us know if some portion of the instructions are hard to follow or do not work as you move through it. We are always looking to update and improve the usability of the data.

So Check It Out!

The Traveler’s Lab Hits the Road

In March, four Traveler’s Lab profs hit the road to deliver papers that drew (at least in part) on research done in the Traveler’s Lab.

Prof. Franklin-Lyons and Professor Jasper both traveled to the Medieval Academy of America annual meeting at the University of Pennsylvania and delivered presentations in a digital humanities lightning round on March 8: Building a Digitized Travel Database: Gazetteers and Roads (Franklin-Lyons) and Teaching Digital Methods in Historical Research (Jasper).  And Katie Jasper brought students with her to help with the open table presentations after the lightning round!  Thanks to Elizabeth Harkert, Mersi Curtsinger, and Logan Janicki.  They did a great job describing their projects and experience working with the lab.

Prof. Gary Shaw traveled to the University of Leeds to deliver the Institute for Medieval Studies Open Lecture on March 19: The Ethics and Politics of Comparison in Later Medieval England. On March 21, he led a University of Leeds faculty group through a workshop on the Travelers’ Lab itself as a possible model and source for inspiration for ways to combine pedagogy and research.

Prof. Jesse Torgerson traveled to Harvard University to deliver the Harvard Committee on Medieval Studies and the Mary Jaharis Center for Byzantine Art and Culture Spring lecture on March 28: Dropping a Medieval Chronicle (and Putting it Back Together): Reading the Byzantine Chronography of Theophanes and George Synkellos through Manuscripts, Maps, and Text Analysis. The “Maps and Text Analysis” portions of the talk were drawn entirely from Traveler’s Lab work done over the last 18 months.

Congrats, and many thanks to the students and teams who contributed: Rachel Chung, Nate Krieger, Jesse Simmons, Yuan Sun, Lydia Nuhfer, Claire O’Pray, Miki Yang).

 

Adam Franklin-Lyons, Elizabeth Harkert, Mersi Curtsinger, and Logan Janicki at the table talking with other medievalists about the travelers lab in Philadelphia (Katie Jasper is taking the photo).

 

 

Lab Meeting: March 4 2019

Lab meeting Monday March 4; USDAN 136 – 12.00-1.10.

This meeting is a (very important) practical discussion on how our different teams are organizing materials for the Great Github Migration of 2019.

Lab members: make every effort to be there as we need everyone’s input on making this work: trouble-shooting, and thinking through potential problems and ramifications of our practices, as well as anticipating how best to accomplish what we want.

Pre-Meeting this is currently understood as
(1) a collaborative space for our teams to work (both at Wesleyan and Marlboro, but also the larger network elsewhere)
(2) a public (“outward-facing”) hub and repository where we can share our data and facilitate collaborative and crowd-sourced aspects, additions, and spin-offs to our projects.

Itineraries, Gazetteers, and Roads – a newsletter article

At the end of last year, I wrote a short article for the Newsletter of the Association for Spanish and Portuguese Historical Studies.  The article talks about our ongoing itinerary project, focusing both on some of the itineraries I have been working on in Spain, but also talks about the broader collaboration between Wesleyan, Marlboro, and Illinois State that has produced the data.  There are also a few visualizations comparing Roman and Early Modern road use by the royal entourage in the thirteenth century in the Crown of Aragon.  It’s available Here at the ASPHS website – the edition is Volume 9 (2018).

 

Adam Franklin-Lyons

One example from the newsletter of short trips taken by king Jaume II around Valencia showing his use of newly constructed road systems instead of older Roman routes.

Lab Meeting: Jan 28, 12pm

Welcome back to the Spring 2019 Semester!

Our first (informational and organizational) meeting for the year will be Monday January 28 at 12-1pm in USDAN 110.

Lunch will be provided!

We will be discussing ongoing projects, ongoing needs, specific semester plans, and how those semester-specific plans fit into the larger trajectory of the Lab.

Anti-Jewish Riots and Royal Communication Networks in Aragon

Royal Communication Networks in Aragon During Anti-Jewish Riots : 1391-1392

by Kaitlyn Thomas-Franz

Introduction

This past semester I had the opportunity to work with Professor Franklin Lyons on an exploratory data analysis project relating to a series of anti-Jewish riots that occurred in the crowns of Castile and Aragon between 1391 and 1392.  The initial intent for the project was to gather as much information as possible for each city where a riots were recorded as having taken place, generating a relative timeline of events relating to the riots for each of the cities.

Method

Starting our data collection by focusing of the territory of Aragon, I performed a close reading of both Benjamin Gampel’s book Anti-Jewish Riots in the Crown of Aragon and the Royal Response and Abigail Agresta’s article on the riots that occurred in the city of Valencia, “‘Unfortunate Jews’ and Urban Ugliness: Crafting a Narrative of the 1391 Assault on the Jueria of Valencia.”  Noting each line of text where a date was given for a particular event, I recorded each of these incidents on an Excel spreadsheet.  I also tagged each events using the following categories:

riot_1391
legal_pre
legal_post
violence_post
violence_pre
communication
travel_royal_official
travel_king
and
travel_city_representative.

The first label notes the date or dates when rioting occurred in a city.  The “legal” labels refer to judicial actions that occurred either before or after the riots, including legal actions against Jewish communities and the arrest/execution of rioters.  In connection, the “protection” label refers to any official decrees specifying the protection of the Jewish community of a city independent of judicial punishments.  The “violence” labels note events of violence committed either against the Jews or against a city that are distinct from the riots themselves while the “communication” label notes letters sent between royal and city officials.  Finally, the “travel” labels note the movement of royal and city officials across cities.

Network Analysis: Preliminary Network

When I finished my preliminary timelines of events for Aragon, however, I noticed that  the majority of the information about the events was derived from letters written between royal and city officials.  Thus, Professor Franklin Lyons and I decided to narrow to scope of the project for the time being to a network analysis of the communication webs to see what sort of information could be gathered about the institutional response to the riots in relation to different groups’ geographical jurisdictions.  In all, the network I generated is comprised of 291 letters from King Joan (the ruler of Aragon), Queen Iolant (his wife), and other Royal and City Officials, with a main focus on communications being between July 1391 and April 1392, which is the month in which major rioting ceased in Aragon.  Using Gephi, I generated a preliminary network that I then color-coded based on its modularity centrality, the number of distinct clusters within the broader network.  As seen below, the network is comprised of three distinct communications groups: King Joan, Queen Iolant, and Duke Marti and Duchess Maria (Joan’s brother and his sister-in-law).

Networks: Duke Marti and Duchess Maria

Curious as to how the sub-networks of each of these royal figures connected to their respective jurisdictions, I generated a series of maps in R that reflect the different regions to which King Joan, Queen Iolant, Duke Marti and Duchess Maria sent communications and noted the content of each geographically-based link.  Below I have feature examples of the networks I created for Duke Marti and his wife.

 

 

 

 

 

Image: Map of Duke Marti’s communications network

Present in Valencia when riots occurred in the city in July, Duke Marti remains there for the remainder of the year.  As the crown prince, Marti had direct responsibility for the city, which is a potential reason why there are so few communications from King Joan to the city during the first month following the rioting.  Marti was also the first duke of Montblanc, which explains his communications with a city far from his purview during the rioting activity.  Nonetheless, the network also highlights that Duke Marti maintained extensive communications with both his brother and his wife, thereby suggesting a more unified royal reaction to the rioting occurring in the city.

 

 

 

 

Image: Map of Duchess Maria’s communications network

While the duchess resided in Tarrega, Catalonia for much of 1391, she maintained consistent communications with cities in the northern section of Aragon.  Because her father was the lord of Luna and the other two regions were under the jurisdiction of her husband, her contact with those cities is evidence of her use of family networks as a means of both relaying information about the riots to those cities as well as attempting to prevent rioting in those regions.  Furthermore, this network demonstrates that Maria shared equal responsibility with her husband in the operation of their jurisdiction, acting as his political representative when he was occupied by the rioting in Valencia.

Conclusion

This project was a valuable introduction to the Traveler’s Lab, allowing me to learn how to use both R and Gephi to generate intricate communication networks.  In the future, I hope to continue with this project by expanding the number of featured communications as the majority are from King Joan and other royal officials as opposed to city officials.  Furthermore, I hope to gather additional sources in order to develop similar event timelines and communications networks for the rioting activity in the crown of Castile.

Lab Meeting: December 10, 2018

The Traveler’s Lab end-of-semester meeting for Fall 2018 took place on Tuesday December 10, 12-2pm in Allbritton 304.

We heard from Kaitlyn Thomas-Franz (working with Prof. Franklin-Lyons on Royal Communication Networks in Aragon During the Anti-Jewish Riots of 1391-1392, from Prof. Gary Shaw (below) with an update on his project to map monastic houses and hostels as a hospitality network in late medieval England, and from Nate Krieger, Yuen Sun, and Ezra Kohn about their work on mapping the geographic references and key actors in the ninth-century Chronicle of Theophanes the Confessor.

Lab Meeting: October 12, 2018

On Friday October 12 the lab met 2.30-4pm, in Allbritton 004, for a University-funded visit from Prof. Kathryn Jasper.

Dr. Jasper’s visit highlighted the lab’s commitment this year to increasing multi-campus collaborations. This semester Traveler’s Lab students working with Prof. Franklin-Lyons and Prof. Torgerson teamed up with Prof. Jasper’s students at Illinois State University to share resources, projects, and best practices.

This is the first opportunity we have taken to work so closely with students who are not at Wesleyan University or Marlboro College, and so Prof. Jasper’s visit allows us to look at how well the collaborations are going, to confirm plans for finishing the semester, and to learn more about her own research project.

Prof. Jasper shared from her own research using analytical & visualization tools (especially ArcGIS) for historical research into how the “monastic network” of one reform-minded eleventh-century monk with a more than healthy dose of ambition — Peter Damian — came into being and continued to function in practice.

Lab Meeting: September 25, 2018

Our second meeting of Fall 2018 took place Tuesday Sept 25, from 11.50-1.10p in Allbritton 004.

Prof. Pavel Oleinikov presented a workshop on Extracting Place Names from a Text using a bit of R coding. This was an exciting opportunity for all of us to start actively sharing methodologies. It was quite an adventure in getting a number of us with little no coding experience to try our hand!

Image: Prof. Oleinikov breaks down the logic of his named entity recognition code.