top of page

Augmented and Virtual Reality for the Humanities

shakespearevr.png
Elsinore Castle From Afar
The rampart...Hamlet meets the ghost.

For the past year, Kevin LaGrandeur and I have been developing a concept for an augmented reality application to help teach difficult humanities texts. Since we are both English professors, the likely place to start was literature. In our experience, students often struggle with the complex historical and cultural contexts with a play like Hamlet and this serves as yet another barrier to reading.

Most teachers of the play can speak to the complaints regarding the language; there are tools to help with that. Imparting context through visuals and explanations remains more complex. Often, videos are used and this can lead to student reliance on this medium in place of reading. Our goal is to compliment and encourage student reading, not detract from it by providing an easier avenue.

Initially, we enrolled in a National Science Foundation boot camp based on the merits of our concept. Though enlightening, this did not prove a viable avenue. With funding from NYIT, and help from several companies, we have embarked on creating a prototype of our app, one which focuses on the appearance of the king's ghost in Hamlet. We want students to see the importance of the clothing of both the king and Hamlet, along with the cultural context of a deceased king haunting his former country.

This will happen through student interaction with a 3D model with their phone. In augmented reality, they can place the model and then zoom in on key areas within. Tapping on the ghost will reveal key information on the former king, his armor, and the probability of his existence. Doing so on Hamlet will indicate his disheveled clothing, and how this may indicate his state of mind. There will be other actionable areas about the rampart and castle, but for now, these two are the focus.

Update 9/15/2018:

We have decided to employ gamification elements into our application and help lead students toward a decision: Does Hamlet truly see the ghost of his father, or is this an illusion created by his own mourning and melancholy? Students will make choices and these will lead them to score points which will indicate, at the end, an implication of truth or delusion.

We have worked with Dell and Enduvo to acquire software to make the transition from modeling software to virtual and augmented environments. The road to the application is forked between virtual and augmented reality. Our initial idea was to use AR because it does not require special hardware and is cost-effective. Enduvo plans on implementing AR in their software but does not currently support it.

Update 10/5/2018:

Enduvo has been a great help, but getting 3D models ready for export has been a major roadblock. We do not have any coders yet for this purpose, and learning these skills on the fly takes a lot of effort. It's like learning another language, except that with spoken languages, the listener can interpret and understand misspoken sentences. A computer language with mistakes just kicks back errors, or, worse, nothing, with no way to disseminate the problem(s).

Update 10/20/2018:

With an updated storyboard and concept, I have made progress in creating a workable 3D model of Castle Elsinore, the ghost, and a poor example of Hamlet. They have all exported into Enduvo as well as Unity and Unreal. The previous problems with textures and materials have been overcome, and I have created a mockup video of how the app should perform. It's skeletal right now (no undead pun intended) but it is at least progress. 

I have shifted from Sketchup to Blender for the models, and the ability to actively render in the latter program has helped, as has the easier interface for the inclusion of lighting. Who knew that the biggest problem I would have could be remedied by digitally flipping a switch?

Update 10/22/2018:

BREAKTHROUGH!

After literally months banging my head against the wall, I have come up with a way to animate and export video from both Sketchup and Blender into a VR app. I've also learned how to work with scenes in Sketchup to create an animation that I've screen recorded. This now gives a clearer view of what the app would look like. The VR stuff is nice, although I see that as a secondary product. We still want AR, and once I find a viable outlet for that, I will work on tweaking the model and animation for that platform.

Update 10/25/2018:

A few more major breakthroughs. The castle and rampart animations are now near complete and look much better. Although most of this is still rough, it looks good for a prototype. The goal is to allow students and professors to get a feel for what the app might do, and to tell us what would work and what wouldn't. We are close to a design that can provide that. At least I am seeing light at the end of the proverbial tunnel.

Update 10/30/2018:

Finally, a major step forward again. Now, I have created actionable areas within Sketchup that bring up text alert boxes. It's rudimentary, but it works. Now, someone can tap or click and information on the ghost, Hamlet, the castle, and even a cannon give key information that can help students better understand what is going on when Hamlet 'sees' his father's ghost. It's a mockup right now, but it provides a prototype of what we want our app to do.

I've also switched to the Unreal Engine, as the models from Sketchup tend to work better with this game engine. I've already learned how to code and move characters around, so I am hopeful I can translate what we have into some sort of playable game. This would all be a lot easier if I was a coder, but, alas, I am not. When we get the funds to hire a small coding team, I suspect we will move forward rapidly.

Update 11/5/2018

Another major breakthrough. I've found a new model to use as the base for our concept. This one has better textures and does a fantastic job of creating a visualization of our idea. Now that I have animations down, I really feel like this one flows and gives a good impression of what our game will be like.

Welcome to my website. I am a college English (Humanities) professor and a mystery and childrens fiction writer.

No more Assistant Professor! I am now Associate Professor of English and Director of Technical and Professional Communication at New York Institute of Technology in Old Westbury, New York. I have received tenure as well and this marks a major milestone in my career. I don't feel like much will change but there is a sense of exhale.

Over the last year, I have worked hard trying to modify classes for online learning. I will spend the summer going over course design and finding ways to make online classes and asynchronous elements more engaging. A lot of it is trial and error but I've found involving students from the beginning effective. Polling them and seeing how a specific class is handling this new modality works wonders.

 

Since I started full-time at NYIT, I have published twelve articles, including several on developing video games for learning in the composition classroom. My project Perchance seeks to re-imagine ancillary literature learning tools like Cliff Notes to become a more usable (and more used) form of learning. Check out the development of Perchance: A 3D Hamlet Mystery here.

 

My teaching interests lie in first-year composition, in introducing students to college writing and finding creative ways to make such courses engaging, and technical writing, as I have discovered these courses attract students of all backgrounds. I didn't like boring classes when I went to school, so I attempt to prevent my students from suffering the same in my classroom.  Technology and its impact on our scoiety as well as the teaching of writing intrigues me, and I spend much of my research time devoted to exploring its uses. I have found, for instance, a direct correlation between the use of technological aids in the classroom and improved student writing, particularly for international students and second-language learners. I have written a paper on using virtual reality (VR) to help students experience and better understand sense of place in writing. It will be published in Computers and Composition later this year. You can see some of my other papers over at Academia.edu They range from using smartphones as writing devices to video games as literature and strategies to make writing more accessible to all students.

 

I find teaching writing helps students across their college careers and life. Formulating an argument or just finding a way to say what we want/think effectively improves all areas of our lives. A quick look around social media can easily illustrate how more of us could use some help here. Students know how to engage, and most merely need a better focus on how to do that more effectively.That's what I bring to the classroom: writing instruction with real-world implications that students can relate to. They work on the skills they will need in their careers while in my classroom. The days of using pen and paper are almost behind us, so I shift the emphasis to the media we all interface with daily. Instagram, Twitter, Facebook, and other forms of digital and social media are incorporated into my classes, and students respond positively to their use in writing instruction. Maybe I am helping improve the Twitter-verse one class at a time. That would be a good start.

 

I write murder mysteries and childrens fantasy books and always look to incorporate my experiences writing in these and other genres into my pedagogy. I believe an intersection lies between creative writing and composition, and try to use this tool to make my classes more interactive. There still should be a strong distinction between the two, but I've found some overlapping really helps students put words on the page. With the addition of using mobile technology to have students writing anywhere and everywhere, I've seen a tremendous increase in student productivity. One habit I have is to show students my early drafts, so they can see that writing, even a big project like a novel, does not involve some magical process or comes from only natural talent. I let them see my imperfections so they can realize theirs differ little. They grasp the writing process a little better after I do this.

 

If you've come looking for my book series, you can head to this page, or click the book tab above that can give you a small sample of my writing.

 

Please feel free to have a look around. I'll be adding new things from time to time, and hope you enjoy your stay. You can always contact me at jmisak@nyit.edu

 

John Misak

bottom of page