Episode 7

Published on:

17th Jun 2023

ChatGPT: web scraping and talking OpenAPI specs


Kerri and Ash talk trees! Unfortunately for us all, Ash turns this into a recurring analogy about giving in to our AI overlords.

Talk to your OpenAPI spec

If a tree is going to fall on your house, do you ignore it... or talk to ChatGPT?

Well, Ash is excited about OpenAPI specs that talk back. Our co-hosts talk through a fun approach to this:

  1. Paste in your OpenAPI spec
  2. Ask some yes/no questions about capabilities, correct as needed
  3. Ask some how-to questions, correct as needed
  4. Have fun!*

*Fun = ask for Getting Started guides, look for issues, get it to code, etc**

As part of this, Kerri and Ash talk about learning how to ask good questions:

  • ❌ "What would a Getting Started guide look like?" → (You get an outline of best practices.)
  • ✅ "Write me a Getting Started guide for Node.js 18 developers." → (You get an actual Getting Started guide that a developer could use.)

Thank you AI. We are better humans now.

Writing a scraper with ChatGPT

ChatGPT can't scrape webpages at the moment, but it caaaaan tell you how!

Ash talks about using ChatGPT to write a webpage scraper in Node.js and clobber it together with the Google Books API.

ChatGPT does a great job functionally, but requires some guidance on coding opinions. Ash has coding opinions, so he asks the AI to:

  1. Use ESM imports instead of CommonJS requires
  2. Use async/await syntax instead of Promises
  3. Use Node.js-native fetch instead of axios

Kerri points out how interesting it is that ChatGPT fumbles with ESM imports in the same way that a human would.

Mid-project, Ash teeters on the edge between the theory and reality of an xkcd automation classic... but he triumphs in the end? (He does!)

More fun

  • Kerri mentions that she's looking into a game streaming setup. Ash plans to be the first sub.
  • Ash mentions that he recently spoke at apiDays. The video will be on the apiDays YouTube channel soon.


** ChatGPT advised that:

the bullet point "Fun = ask for Getting Started guides, look for issues, get it to code, etc." could be expanded to better explain what fun means in this context.

But we're confident the real human scripters know what fun means 👾

Listen for free

Show artwork for I'd Rather Be Scripting

About the Podcast

I'd Rather Be Scripting
Adventures in Technology Tangents
Ash and Kerri go on mostly random tangents about technology, music, photography, and just about anything else. You never know what you'll hear!

About your hosts

Ash Ryan Arnwine

Profile picture for Ash Ryan Arnwine
Here! #DevRel human. #JavaScript hacker. #vinyl newb. Life tourist.
 Wouldn't you rather be scripting?

Mastodon-don-don: https://indieweb.social/@ashryan

Kerri Shotts

Profile picture for Kerri Shotts
Principal Product Manager for UXP @Adobe, #JavaScript fangirl, Technical writer, Photographer, #Trans (she/her), #Atheist. All opinions are my own.

On Mastodon (@kerrishotts@mastodon.cloud): https://mastodon.cloud/@kerrishotts