How I built an AI powered Inspection App for the Avengers

How I Built an AI-Powered Inspection App for the Avengers

Even superheroes can’t save the universe without efficient business processes! 

In this Video, I walk through how Google’s hidden superpower, AppSheet, helped the Avengers make reporting on Hero Work a breeze.

This video features a clean & easy Inspection App with a Gemini/Vertex integration. Here’s some of the highlights of the build. If any of these seem valuable to your use cases, peek under the hood and retrofit. 

• Data lives in AppSheet Database; Allows two User types to file multiple Inspections on multiple Sites (with mapping support)
• Virtual Columns on the Sites table Averages the severity score from related Inspections and powers a format rule
• Inspections collect an Image and a Severity score. 
• Bot takes that Severity score and converts it to a Text-Based Generative AI prompt.
• A VERY SIMPLE AppsScript function manages the communication between AppSheet and the VertexAI endpoint. 
• AppsScript file has all the Variables you need to update for your own use marked in a dedicated file. EASY.
• VertexAI returns a dramatized “Mission Report” Summary in Markdown format so it can look clean in the AppSheet Markdown supported LongText field. 
• I have a Loading GIF that shows/hides while waiting for the script to finish. 

‍

Obviously, this use-case is fictitious and I have no personal or professional relationship with the Avengers… 

‍

How to copy and run in your environment

In order to get this VertexAI integration running, you’ll need to follow these steps (The UI / location of some of these things may have changed since writing):

‍

GCP Steps

1. Create / Open an Google Cloud Project at https://console.cloud.google.com
   1. Take note of the Project ID for this project. You will need it later. 
   2. You will need to be broadly permissioned to use the Authentication method I use in the script. 
2. Attach a Billing Method
   1. Search “Billing” in the top bar of the Console. 
   2. Add a credit card to the account. 
   3. The model I use text-bison is SUPER CHEAP. You will likely not spend more than a dollar experimenting with this. 
3. Enable the Vertex AI APIs
   1. Search “Vertex” in the top bar of the Console.
   2. Find the Enable Recommended APIs button. 

Whew… done with that. 

AppsScript Steps

1. Go to my public AppsScript file and Copy it
   1. Navigate to the AppsScript File
   2. On the top right of the information page, there will be a COPY icon. Press that. 
   3. Rename to something local to your environment (You’ll have to search for it in the next step)
2. Change the variable PROJECT_ID in constants.gs file to the project ID your created in the GCP setup. 
3. Run the quickTest() function from the AppsScript Runtime tools (explained in the comments on Code.gs file. This will prompt you to authenticate. Once authenticated, the script will store your access token for a while. This is required for running this code from AppSheet.
4. Update the prompt or the temperature of the AI if you want to explore other outputs 😀

AppSheet Steps

1. Go to my public Sample App and Copy it
   1. Navigate to the link above. Should open the AppSheet Editor for this app
   2. Click the Copy Icon in the top left of the Editor UI. 
   3. Rename the App to your liking. Copy files & data.
   4. You can save to whatever data source makes sense for you. 
2. Retarget the Automation to your version of the AppsScript
   1. Open Automation Menu
   2. Click on the first Bot listed
   3. Click on the Get Summary from VertexAI action step
   4. In the settings of that Step, Click on the Script Project file selector
   5. Retarget to the script file you just created.  
   6. Click the blue Authorize Button
3. Test the app out by adding a new inspection form to any of the Sites

‍

If you enjoyed this demo, had any issues during setup, or actually helped save the world with your adaptation of it, let me know in a comment below!

‍

For more free content, Check out our YouTube Channel. For a big library of Training content prepared by product experts, check out AppSheetTraining.com. 

Do you want a helping hand in unlocking the full power of this technology for your organization? Reach out to us at www.QREWTech.com