This is a site for discussing roleplaying games. Have fun doing so, but there is one major rule: do not discuss political issues that aren't directly and uniquely related to the subject of the thread and about gaming. While this site is dedicated to free speech, the following will not be tolerated: devolving a thread into unrelated political discussion, sockpuppeting (using multiple and/or bogus accounts), disrupting topics without contributing to them, and posting images that could get someone fired in the workplace (an external link is OK, but clearly mark it as Not Safe For Work, or NSFW). If you receive a warning, please take it seriously and either move on to another topic or steer the discussion back to its original RPG-related theme.

Extracting Semantic Data from Classic Modules

Started by Telarus, July 13, 2012, 06:17:45 AM

Previous topic - Next topic

Telarus

So, I have only been lurking for the last few weeks (I'll get back to my Earthdawn-Greyhawk mashup, swear). But during this time I have been learning to use a very cool tool:

V.U.E - Visual Understanding Environment [ http://vue.tufts.edu ]

An open source "Concept Mapping" application that allows you to visually map data. Mostly by hand, as other 'mind mapping' software does, but VUE has some really cool automated tools for importing and extracting data. So, what good is this?

Well, here are the last 10 entries from Justin Alexander's blog (the Alexandrian's RSS feed, to be precise).


https://lh3.googleusercontent.com/-wsAfJej4bjs/UAAHEBqGBnI/AAAAAAAAAjM/fCckBOgwkrs/s800/TheAlexandrianRSS_07_13_12.png

I've dropped all 10 "post" nodes onto the map, then the "hexcrawl" category node. Links are formed due to the meta-data. You can even double-click each node to open that post in your browser window.

[ Then I got fancy and called AlchemyAPI and OpenClais on one post, some web2.0 services, to generate new metadata nodes based on the content of the post! As you can see by the nodes with red outlines, this isn't an exact proccess... it seems they tagged 'Hex' as from the Discworld series.. lol.. Anyway, this step isn't applicable to the use-case I'm leading up to. I just wanted a more complex map to show the relationship features possible with visual relationship mapping. ]



So, what then can we really do with this tool?

Well, obviously we can plot "relationship maps" quickly and easily, with nodes for characters and links for one or two-way relationships.

Even better, once we have laid down nodes and links, then given them labels and keyword-metadata (examples: "NPC", "Treasure", etc), we can use the map-search feature to find unique entities quickly or to extract entities onto a new map (say we need all the NPC's from a certain location, simply search your master NPC database-map for that location tag and generate a new map with all the NPCs tagged with that location name).

Even better (did I say that already?) we can add nodes to other nodes in a nested child/parent relationship, and nodes can take a URI/URL reference. This last features means we can link to any file on our computer OR on the web. The child/parent relationship nesting means we can have multiple resources within a parent node to describe different aspects of it (nodes should be "one cocept" per the VUE specs). A simple doubleclick will then open, say.... the NPC's character sheet (once you've made it, save it, and linked it, of course), or a portrait or token image from your computer or the web (with the ability to preview images in the map itself). Or a PDF file. Or an online Wiki entry...


Have I blown your mind a little yet? (Good, Eris hands out gold stars for that... -><-)

How about this: VUE is FREE AND OPEN SOURCE.

Another one? Ok, so I had to introduce you to the tool for you to understand the next part. This is my main use case, currently:

VUE can import spreadsheet data as CSV files (comma-separated-value text files). So, I am current crawling through Hommlet [T1-4] and jotting down every single NPC listed into a GoogleDocs spreadsheet according to the following Keywords. Who says Hommlet has to be boring? It's just the info-overload that kills it. There's a lot of intrigue and skulduggery in Hommlet, all buried in the Map Key... so we need some way of extracting that semantic data (meaning) behind the repetitious Key.

I've listed these Keywords at the top of my spreadsheet, VUE takes the first row as the meta-data categories for each "row-object" which defines an NPC:

Name: [for unnamed characters use Title(key##), examples: Farmer(01), Farmer's Son A(01), Farmer's Son B(01), etc... this will be our Master Key so they have to be unique]
Location: Hommlet [T1-4]
[Key]: [Location key [##] with leading zero, for sublocations use [##] ## format]
Class:  [examples: commoner, Fighter, etc]
[Hit Die]: [examples:
  • , [2], etc]
Gear: [the npc's equipment]
Loot: [the NPC's carried valuables... NOT the location valuables]
Militia: [militia = a member of hommlet's militia, civilian = not, other = other]
Faction: [so far I have: Old Faith, New Faith, Rufus & Burne, Lareth's Camp, The Temple, Adventurers, Random Encounters ...not all may apply to hommlet]


Once I have this entered into my spreadsheet, I can drop a pic of the Hommlet map into a background layer in VUE and start laying out the NPCs. I can either auto-cluster them by Faction (If I define that relationship as the Key), or by Map Key## (ditto). I can then make a spreadsheet for the location information, import that and drop those nodes (using a different color scheme). Then, I can associate the two datasets, and auto-link NPCs to their locations.


So, is this something that you all would be interesting in seeing in action? Also, what would be some good Column Headers for my NPC roster?
Have I missed any key 0/1e "metadata" which all NPCs usually have? (Besides the 6 stats..I'm not including stats at the moment.)

Example: If I use an "Image" column, and populate it with URI/URLs from the web or my computer (and set the 'image-key' to that field when I import the dataset), VUE will then grab those images automatically! So, I've been considering that on a second pass at the NPCs. I know there are pics of Rufus and Burne floating around out there....


Hope this was useful, stay tuned to see my results. :cool:

The Traveller

Quote from: Telarus;559529Well, here are the last 10 entries from Justin Alexander's blog (the Alexandrian's RSS feed, to be precise).




I've dropped all 10 "post" nodes onto the map, then the "hexcrawl" category node.
I think you've posted that image in a private gmail message or something, can you rehost it on a free image sharing site? Its quite hard to understand what you mean without visual cues.
"These children are playing with dark and dangerous powers!"
"What else are you meant to do with dark and dangerous powers?"
A concise overview of GNS theory.
Quote from: that muppet vince baker on RPGsIf you care about character arcs or any, any, any lit 101 stuff, I\'d choose a different game.

Telarus

#2
Quote from: The Traveller;559532I think you've posted that image in a private gmail message or something, can you rehost it on a free image sharing site? Its quite hard to understand what you mean without visual cues.


Definitely, sorry about that. Well, I'm just past the Welcome Wench and have over 50 characters in my roster (including all the young children).

[Edit: re-hosted out of my picasa album, and added the link..]

Telarus

#3
Ok, I've got a "first pass" done. No color coding (yet), but I have bolded the names of the militia members, and hidden young children under their mothers node (to save space).

I really like what I'm getting so far. Here's a WIP image (note that I have Elmo selected, and his metadata is in the Info window):


https://lh4.googleusercontent.com/-tQbHcSHxmSI/UAAXBXZuAJI/AAAAAAAAAjw/Q21GO4Ff7Rg/s767/HommletNPCs_Test01.png

That should be a public image on my googleDrive. Let me know if anyone can't get to it, I can re-host a smaller image on picasa (less readable, tho).

The Traveller

Quote from: Telarus;559540That should be a public image on my googleDrive. Let me know if anyone can't get to it, I can re-host a smaller image on picasa (less readable, tho).
Nope, can't see it. Just bung it up on postimage.org or one of them, it won't last long but we'll see it anyway. I have to admit after looking at that I'm not much the wiser. Its a relationship map of some sort? And it goes through documents looking for related words?
"These children are playing with dark and dangerous powers!"
"What else are you meant to do with dark and dangerous powers?"
A concise overview of GNS theory.
Quote from: that muppet vince baker on RPGsIf you care about character arcs or any, any, any lit 101 stuff, I\'d choose a different game.

Telarus

Quote from: The Traveller;559542Nope, can't see it. Just bung it up on postimage.org or one of them, it won't last long but we'll see it anyway. I have to admit after looking at that I'm not much the wiser. Its a relationship map of some sort? And it goes through documents looking for related words?


Weird... but the 1st image came through when I replaced it? Ok then, Google+/Picasa it is for this forum. Fixed my previous post.

And here's a slightly bigger version via postimage.org (thanks, hadn't heard of them):

http://s9.postimage.org/bj8j5thu7/Hommlet_NPCs_Test01.png

Telarus

I hope that since no-one else has replied that the images are showing up, & if I'm coming across as too confusing (hail eris?), I'm happy to field questions...

Would a visual key like this be useful to any of you (as a GM) if you re-ran this module (or one similar)?

You can move around the map, zoom in/out (and move nodes around), and run searches on the main map to generate a 'new map' with just the nodes/NPCs you want.

Personally, I find scanning for info much easier in a visual layout with relationships drawn out (and, as the VUE team says, this is based on the last 40 years of visual learning research). I'm testing out my process here, but I think this is a novel way to present adventure information (and you can always include a PDF with the "meat" of the adventure and auto-link to it in inside the map). You can also bundle up the map and all linked resources into one (\zip based) vue-package. Since VUE is free and open source the end user can grab it easily, and you can publish your map as a PDF or HTML image-map to post to people who don't have VUE.

Kuroth

Quote from: Telarus;559529I know there are pics of Rufus and Burne floating around out there....

They are on the back cover of the green edition of village of Hommlet.

It's interesting to pull role-play games into professional tools.  I have used certain professional research techniques before, but one of the charms of such games is to remove one from work-a-day. The things I look for in tools for role-play games are ways to help the in game experience.  It is useful to organize the creative process that one may perform in preparation, but I find that type of organization tends to create a type of target fixation.  Lately, I have set a pretty short time limit on the pregame aspects, which helps make campaigns more lively.  

If the goal is to create new publishable material, it would be different than campaign preparation, and I would certainly try such a process for a writing team that is having difficulty working together.  All published adventures have a greater or lesser level of fixation upon certain goals or outcomes.  So, the issue is less of an problem there.  

Good work over all.

John Morrow

Quote from: Telarus;559540That should be a public image on my googleDrive. Let me know if anyone can't get to it, I can re-host a smaller image on picasa (less readable, tho).

FYI, I can see your images.
Robin Laws\' Game Styles Quiz Results:
Method Actor 100%, Butt-Kicker 75%, Tactician 42%, Storyteller 33%, Power Gamer 33%, Casual Gamer 33%, Specialist 17%

Black Vulmea

Quote from: Telarus;559529Well, obviously we can plot "relationship maps" quickly and easily, with nodes for characters and links for one or two-way relationships.
Does the software plot the lines between characters itself. or do you have to draw them manually?

One of the problems I've had with using software like this for relationship-map-types-stuff is that when a character has thirty or forty relationships - think Cardinal Richelieu in my Flashing Blades campaign - then it becomes a real chore to manage. If the software automatically finds a best fit line, that would be a huge time saver.
"Of course five generic Kobolds in a plain room is going to be dull. Making it potentially not dull is kinda the GM\'s job." - #Ladybird, theRPGsite

Really Bad Eggs - swashbuckling roleplaying games blog  | Promise City - Boot Hill campaign blog

ACS

Telarus

#10
Quote from: John Morrow;560003FYI, I can see your images.

Hi John. Thanks, good to know. :)

Quote from: Kuroth;560002[...snipped very useful feedback...]
Good work over all.

Thanks Kuroth, that was very useful commentary. I have used VUE as a "taking notes" replacement while in class, and once you get used to the interface it's as fast if not faster than hand-written notes. I wanted to experiment with this as a GM tool, and as a way to present new Adventures/Modules. I thought crunching through a complex area like Hommlet would be a good way to see if there is a benefit from using the tool.

Quote from: Black Vulmea;560008Does the software plot the lines between characters itself. or do you have to draw them manually?

One of the problems I've had with using software like this for relationship-map-types-stuff is that when a character has thirty or forty relationships - think Cardinal Richelieu in my Flashing Blades campaign - then it becomes a real chore to manage. If the software automatically finds a best fit line, that would be a huge time saver.

Hi BV. This is what I am experimenting with currently. My first data-set was basic NPC notes. My next one will be who the spies and secret agents in Hommlet report to. It will be laid out similar to the example below (and I can "associate" the two datasets via the Name field to link them, so VUE knows that those nodes are "the same").

VUE supports importing spreadsheet-style tables (in CSV format), and one of the options is called "Matrix Data". There are "Tall" and "Wide" matrices, but this just describes how you've laid out your data in the spreadsheet.

Let's take a "Tall" example. The spreadsheet would look like this (in CSV format):

Name1,relationshipTag,Name2
Person1,really hates,Person2
Person1,is related to,Person3
Person2,really hates,Person1
Person2,is in love with,Person3
Person3,is related to,Person1
Person3,is in love with,Person2

Then, once you've imported it correctly, you can just drop all the nodes on your map and it will link them. I believe it will also update existing nodes if you update your CSV file and refresh the dataset window in VUE (still need to test this out). Here's a quick image of the example spreadsheet in vue. The links markd with <> are auto-generated. All I have done is 1) Dropped the nodes to the map. 2) Spread them out a little bit. 3) ran a search for "really hates" (this auto-selects the links with that tag) and changed the link color and line-weight.


Telarus

#11
Quote from: Telarus;560016I believe it will also update existing nodes if you update your CSV file and refresh the dataset window in VUE (still need to test this out).


Just tested this, and it seems like the links won't auto-update, but if you delete the changed node and re-drop it, the new links show up. Not too hard, but it would be better if links could auto-update like the nodes. I'll have to check with the VUE programmers about his (although the forum hasn't been active recently). Might be easier to just change the link labels in the VUE map when the relationship changes... although I would prefer the update functionality.

Black Vulmea

Quote from: Telarus;560023Just tested this, and it seems like the links won't auto-update, but if you delete the changed node and re-drop it, the new links show up. Not too hard, but it would be better if links could auto-update like the nodes.
Yes, it would.

Thanks.
"Of course five generic Kobolds in a plain room is going to be dull. Making it potentially not dull is kinda the GM\'s job." - #Ladybird, theRPGsite

Really Bad Eggs - swashbuckling roleplaying games blog  | Promise City - Boot Hill campaign blog

ACS

RPGPundit

I can't understand what the point of any of this is.

RPGPundit
LION & DRAGON: Medieval-Authentic OSR Roleplaying is available now! You only THINK you\'ve played \'medieval fantasy\' until you play L&D.


My Blog:  http://therpgpundit.blogspot.com/
The most famous uruguayan gaming blog on the planet!

NEW!
Check out my short OSR supplements series; The RPGPundit Presents!


Dark Albion: The Rose War! The OSR fantasy setting of the history that inspired Shakespeare and Martin alike.
Also available in Variant Cover form!
Also, now with the CULTS OF CHAOS cult-generation sourcebook

ARROWS OF INDRA
Arrows of Indra: The Old-School Epic Indian RPG!
NOW AVAILABLE: AoI in print form

LORDS OF OLYMPUS
The new Diceless RPG of multiversal power, adventure and intrigue, now available.

Telarus

#14
Hi Pundit. Let me see if I can answer that for you.

Quote from: 'T1-4 Hommlet - 15. The Moneychanger'[After specifying about 20-25K in raw coinage, jewels, and bar-metal stock.]

Be sure to keep track of all transactions made. The stock in trade must be correct, for example gems cannot be obtained when Nira has exhausted his supply. He can purchase more from merchants, at market value, every 7-12 days.



Ok, ok.

So I'm running T1-4 and at the same time converting it to Earthdawn 3rd Edition on-the-fly, which means translating D&D terms into Earthdawn terms (where all high level characters channel magic to walk their path). My rough scale right now is every 2 D&D 'levels' = +1 Earthdawn 'Circle' for an Adept (similar scale with HD and monsters). Also, I did a simple down-shift on the currency denominations to align with the Earthdawn 'silver' standard (gp becomes sp, etc..).

I ran the first few sessions on 3x5 Notecards and graph paper and dice (and a few map images for the players). Multiple delves into the Moathouse so far, and Lareth's pissed.

I have info overload, man.

And here Gary or Dave is telling me to track each individual cash-in at the moneychanger! (That fits well with the Treasure in GP -> XP idea, really...) And they had multiple adventuring parties crawling around, right?! Man, I'd need to invent my own card catalog system.

So, I found a different way to keep my campaign notes.

It's a searchable digital whiteboard. Which can import spreadsheets.

I'm really just starting to experiment with it.



So, I finished my first crawl though of the Hommlet map key. I've plucked out the details of 210 individual NPCs! (Wow Gary...) Plus the transient Hommlet population (+1d12 "laboror camp followers" i.e. women and children, and 1d6+1 travellers and merchants at the Inn of the Welcome Wench).

Hammlets total population (before the PCs show up and start upsetting the delicate balance) is roughly 213-229 individuals.

Cool! I'm going to start laying this map out and will have access to over 200 npcs' basic stats and notes on "one page", which I can modify on the fly during game.