Diverse, Unbiased Data Sets are Essential for Shaping the AI of Our Future

Note: This is a preview of a talk I will be giving at the Women Who Code Connect 2019 conference.

In the new information economy, data is more valuable than oil. It’s essential that as developers we’re aware of how data is being used in the tech products we build. Our book app, MyLibrarian, is powered by a diverse data set and is an example of curation by unbiased experts for better recommendations and more inclusive AI.

At In the Stacks, we’ve created an app for book discovery. In recent years, we’ve been building and scaling our product, and we’ve focused on creating a data set that will shape the way book lovers get their book recommendations. Instead of asking friends what to read (and risk staying inside a biased bubble of people who think alike), why not ask unbiased book experts, whose job it is to find books that will change reader’s lives? When Librarians choose your next read for you, you’ll get a diverse list of book recommendations, based on descriptive keywords the user chooses, and how highly librarians rate the book.

We’ve worked on building this product as a part-time project for several years now, and have found that creating a data set, who we call the Librarian Brain, and which generates fresh new book picks, is valuable. Most online booksellers still rely on cross-selling recommendations, or reviews from friends and family, or even paid reviews. Our product is shaped by a data set—tens of thousands of books librarians love—with a custom-designed taxonomy. Our data includes the best book industry keyword descriptors, to create an experience similar to asking a librarian at a library reference desk what to read. A larger book catalog is then run through our set, to provide readers with five book picks. We provide outbound reader’s advisory—bringing that experience out of the library, and online—in an app.

Creating data sets to offset bias the we’re seeing in ML/AI is essential. At least some human-curation is beneficial, and librarians are ideal for this kind of work. Librarians are a progressive, inclusive group, who all have Information Science masters degrees, are technical, and sensitive to bias and discrimination. Plus we are detail-oriented data nerds. Librarians are champions of banned books, free speech and marginalized groups. We promote diverse books, and our recommendations help to offset the often unintentional partiality and bias inherent in some products, and raise aware of the quality of data used in fueling the products we’re coding and designing.

A Little More Background

For the last 4 years, our team has been researching and searching the book review sphere for the best reviews, and scraping the open source data from these reviews into a database. We named our database The Librarian Brain, because the reviews are all from smart librarians who have graduate degrees.

We debuted an early version of the app, an internal alpha working locally, at a demo day in 2016 (view the program here: https://drive.google.com/open?id=0B-Ug9haIJqKmTFQ0SF9nYzE3QWc). Since then, we’ve spent time working on the business model, through programs offered by the NASDAQ Entrepreneurial Center SF, landed on a B2C business model, and have shown our demo to several online booksellers. Our data set currently runs through the open source book corpus from Internet Archive.

This past year, I’ve pitched many investors, and have created relationships with some who have become mentors. We’ve raised about $25K, through crowdfunding and other sources and are still bootstrapping.

I create all aspects of the product, build what I can and oversee the execution of the remainder of the product with our team of 8+. I’ve created the data set taxonomy and product UI and functionality and my contractors improve on it. Timing is everything with this product—I’m glad we have more of the business and marketing plan in place before the app is in the stores. If you’re interested in hearing more and viewing our pitch deck, get in touch at info@inthestacks.tv—Michelle Z.

p.s. Here’s a paper about another human-curated data training set we worked on, Applying Narrative to Metadata for News

Bio

Michelle Zaffino is founder of the book discovery product MyLibrarian, whose app and book club service launches in 2019. The author of five books, Michelle realized that whether your book is indie or traditionally published, book discovery is broken, so she learned to code in front-end web development bootcamp, in order to build products to fix it. MyLibrarian / In the Stacks is our way of sharing good books with the readers who love them. The Librarian Brain data set was born out of our love for metadata and aims to ensure diversity and inclusion in book publishing artificial intelligence. When she isn’t working remotely as a virtual nomad, you can find Michelle at a coworking space or coffee shop in San Francisco, where she’s lived for 15 years.
http://www.michellezaffino.com/diverse-unbiased-data-sets-are-essential-for-shaping-the-ai-of-our-future/
www.michellezaffino.com
crawl=CC-MAIN-2019-47
63517
application/http; msgtype=response
sha1:CFFEV4E3LSGNPSULY7SYB2AQZVO6EHHI
<urn:uuid:22c4abcd-e1e3-4fac-a134-4b261d534e70>
2019-11-22T14:37:06Z
184.154.2.18
text/html
sha1:RZCF2S7SJQQVCJO4HLYHDV53ZGGEL75U
<urn:uuid:ae11c756-24fc-4b90-89fd-aa1cbf06ef71>
http://www.michellezaffino.com/diverse-unbiased-data-sets-are-essential-for-shaping-the-ai-of-our-future/
response
<urn:uuid:a52a6478-4720-4322-95a5-fc3a394eec39>
isPartOf: CC-MAIN-2019-47
publisher: Common Crawl
description: Wide crawl of the web for November 2019
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-67-67-135.ec2.internal
software: Apache Nutch 1.16 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 1.1-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: http://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/