This post continues a small series I’m doing on the AI Competencies for Academic Library Workers. Continuing my contrary tradition of going out of order, we’ll be looking at competency area 3 in this post: Analysis and Evaluation.

Evaluating Output and Evaluating Tools

Being a librarian, when I first read “Analysis and Evaluation” I immediately thought of source evaluation, a topic that librarians teach frequently. Source evaluation involves looking critically at a piece of information and determining whether that information is factual, reliable, worth using for a research assignment, or worth sharing on social media, as the case may be. This sort of evaluation is a bedrock of information literacy skills.

But I was pleasantly surprised to see that this section of the competencies was actually talking about evaluating AI tools themselves in order to “critically assess their application.” In other words, asking questions such as “Does this AI tool work? Is it useful? Are there benefits or harms associated with using it?” Or is this AI even an AI or rather a group of humans pretending to be an AI, in the equivalent of a person dressing up in a robot costume and saying beep boop? True story actually! If you aren’t familiar with the wild tale of Builder.AI, please take a moment to familiarize yourself with the brazen (and to my mind, somewhat hilarious, albeit in an awful scammy way) scheme, which I mentioned briefly in my first ever post on this platform. Fond memories. Anyway, moving right along.

Continue reading over at my Substack site and feel free to subscribe if you’d like to receive updates to your inbox!