Jev AI in Maia Pipelines: Choice, Score and Noul Evaluations
Jev AI makes fast, structured decisions with far lower token consumption than a traditional LLM. This demo shows how Maia builds and updates pipelines around the three Jev question types: Choice, Score and Noul.
Walk through support ticket triage, content quality scoring, binary compliance checks, product review analysis and customer churn, each running through the same shared pipelines. Watch Maia update a live pipeline to add a new docs department, then re-run it and sample the results from Snowflake.
This video is part of the Building with Maia and Jev series. Read the full write-ups:
Hi. My name is Joe, principal solution architect at Maia. Today, we're gonna deep dive into how Maia can build Jev pipelines to handle all of the different scenarios in which Jev's able to help with decisions in a much faster and much less token consumption way than say, your traditional LLM models do.
First of all, let's have a look at this manifest. It's a really exciting manifest. So we ship the shape of the machine native, composable AI with the highest possible intelligence per dollar and make AI reliable enough to transform the economy. These are really the two points I'm gonna focus on in the demo in terms of how we can leverage Jev to help our customers and help them build, pipelines that take advantage of these new systems, system one thinking.
So first of all, I'm gonna come into Maia Foundation, and I've already been building some, pipelines this morning with the Jev integration. And Maia actually built out the full shared pipelines that you see here. So you can see that we've got these custom connectors that take, either the post of the evaluate source or listing the models, and I've got a token that links out to my, TypeSafe usage.
So you can see here that I've got the ability to configure this connector, inside of Maia, and we're gonna share this as a shared pipeline so that other team members can also leverage this, connector and logic as well. So now that we've got our sort of connector built out via API to TypeSafe, what we're able to then do is populate sort of four different pipelines here, both orchestration about how you might wanna call these in batch or via a table iterator and loop through multiple queries that you might wanna run out to Jev.
And then also the cleaning and sort of unpacking steps where you can get some answers back out from the structured data, semi structured data that comes back from your Jev queries. So these four pipelines are the shared pipelines. These are the kind of composable logic that you can share across your team and you can update here in my sort of remote branch that I have and then also sort of feed out and seed other users of... That want to leverage this logic.
To accompany this, I also built a sort of Jev skill. So in here, can see we've got all of our different skills that we have and the ability to sort of see when this skill might be activated. So when another user wants to come in and leverage, this new capability, we've got some skills here that sort of describe the logic of how Maia can work with Jev as part of building a pipeline that perhaps presents decisions to the Jev models and also leverages the components that are part of the shared pipelines. It's also got some nice anti patterns in here as well.
So let's have a look into some of these, sort of examples. First of all, what can Jev actually do? So here you can see in example number one, we've got an orchestration pipeline. So our orchestration logic takes data from a source, writes it into a target, Snowflake, BigQuery, Databricks, Redshift, and then also calls any subsequent parts of this process. In this example... And all of the pipelines follow the similar format. And, of all, creating sample data and then running it through a Jev model.
Here's some of that sample input from these support tickets. So quite basic support tickets just to begin with, just to familiarize ourselves with what we're looking at, charged twice for subscription, requesting a refund. And so these tickets and IDs are created in a table. They then run through this shared pipeline that I've already demonstrated here, which then is gonna loop through that table via the iterator component, mapping the sort of text and the keys, and we've got a sort of dynamic input that we leverage variables within the pipeline to help us do this at scale.
So you can see some of the pipelines that we have... That some of the variables that Maia has populated down here that help with this sort of looping mechanism. I come back up to this example. This example is gonna create those sample tickets and then run it through this pipeline, and you can see here that Maia has then set the scalar variables to take those ticket columns, the columns that contain the ticket ID, and pass this through this shared pipeline. So this is a shared pipeline that we see here. And you can see that Maia's created this in a dynamic way.
So I'm gonna execute this pipeline now, and we can see... You can see a sneak peek of the results over here on the right hand side. But what I'm just gonna describe is this YAML file, this orchestration pipeline is now running inside of my AWS environment. You can track these pipelines, running here in our tasks. And you can see how far through it gets. So you can see that Maia is now creating that sample data, inside of, in this instance, a Snowflake environment, and then it's gonna run that through the shared logic that takes this out to the Jev model.
The configuration that we have for Jev in this model is working out which department or which team, sorry, should handle this ticket. So if it's to do with payment subscription and refund, that goes to billing. If it's a bug error, integration, or data issue, it goes to technical. And if it's pricing, it goes to sales. And then the second question is, what is the sort of priority level that this should be assigned? So again, you can set all of these parameters up.
This is just kind of what Maia thought was best for this, type of sort of ticketing scenario, but you can set this up and improve this to match whatever logic you'd like. And Maia can help you sort of set and improve that as well. So for instance, if I say, I also want to, identify where there might be support ticket related for the docs team. Update this pipeline and the documentation to accommodate for that.
So Maia is gonna take the kind of current state of what the end user, in this case myself, is looking at on the screen. It knows that this orchestration pipeline is open. It's got the context from the ability to read this YAML file and understand the logic of the pipeline as well. And it's gonna go ahead and it's already actually gone into plan mode, so it's going to then evaluate how it can best update this pipeline to accommodate for the additional criteria of... It's not just billing, technical, and sales. I also want to see documentation in there as well.
So add a docs criteria to the department. Yep. That's fine. Update the Canvas notes, and it's also gonna tell me which components are affected. So, yeah, I'm gonna accept that. So quite a simple sort of basic change to just add in that additional configuration for this example of the support ticket triage. You can see here the outputs that we get back from Jev. So we finished running that pipeline. So Maia's also able to sample the output. It results back from the Snowflake table.
So requiring my permission to then execute this pipeline again, I'm gonna give it permission to execute this pipeline. I manually triggered that here with the run button last time. And Maia is now gonna come back and sort of show me the actual data results of this. So I wanna see an additional ticket coming through that sent out to the docs department team in this instance. So Maia is able to update the logic on the fly of the pipelines that you build, and you're also able to have the configuration of the questions that you want to send to Jev being updated by Maia as well.
So this is kind of just an example of how you might amend an existing pipeline to take advantage of how we go from, support tickets to ingest an additional new docs team that you wanna have updated as part of your integration to Jev. Maia is just gonna come and bring us back this... These results here. It's just running at the moment. Whilst we wait for Maia to run, I'm gonna introduce the second example of taking advantage of Jev's calibrate scoring capability.
So in this instance, I've got perhaps three articles. So these are research articles to do with particular topics that might be of interest to my research team. The questions that I wanna ask is, you know, how clear and well written is this content? And you've got some criteria here and then also technical depth, you know, how technically detailed and accurate is this and I wanna give it a score.
And so in this example, Maia built this pipeline that again leverages the same shared logic of that Jev integration and the output from Jev is here. You can see the sort of clarity score that Jev has presented on each of these articles based on the description, the confidence that it has in that score, and then also the technical depth score and the confidence that it has in that as well. So a really powerful capability for taking a lot of unstructured data, albeit this was just a sort of small, corpus in, you know, a couple of paragraphs in each, but still nonetheless, sort of conceptually showing how you can use Maia to help build content quality scoring pipelines with Jev.
So the third example is this Null, I think I'm pronouncing that correctly, Null questions for binary compliance checks. So this is bringing back a sort of a true or false flag, so a yes or no answer from Jev. So you can see here that we've got does this document, you know, specify an encryption standard for data at rest? Yes or no? Does it have a deletion process, retention process, access controls? And so you can see in this example the document zero one has some information around policies.
Again, this is a synthetic data that's been created here. We've got real data in a second. And then we can see the output here from, Jev which is a score of the confidence based on whether or not the document does contain encryption policies, yes or no. Does it contain retain policies? Yes or no? And the associated, certainty with that as well. So it's nice that it comes back with a numeric certainty, think, and not just like a general state of yes or no capabilities.
And so here you can see when we made that update earlier, there we go. We've got an additional ticket that's been created, API docs outdated, references removed. This is that support ticket triage pipeline. Maia's now finished adding that sample data in and running the pipeline. It's now got us with our results here on the right hand side. So I can see that Maia has now taken that new department of docs. It's left everything else unchanged, and it's given some nice confidence and priority ticketization to that as well.
So Maia is really quick and easy at being able to update these pipelines and not just with the build side as well. So we've got three things. Firstly, the support ticket triage. Secondly, the sort of quality content score. And then third, the sort of compliance check, like, does this document contain these kind of features? So what happens if we put all of this together?
So in this example, Maia's built a sample dataset that has an ID for our product reviews. Love it. Outstanding build, early delivery, great battery. And in the questions that we've built here with Maia is what is the overall sentiment? Is the, you know, are the... Is the customer happy, satisfied, or recommending? And essentially classifying the sentiment of the review into either positive, neutral, negative, or mixed, and some examples of that criteria.
Satisfaction, so what's the satisfaction score of this customer, you know, and what does that look like and, so not just are they recommending, but what is the satisfaction levels and then also does the... Using the null type here, is there any defects mentioned? Is there any requests refunded? Would they recommend sort of yes or no? So if we look at one example, love it, outstanding build, early delivery, great battery. We come over here, positive sentiment, feels right, strong score, no defect mentioned, no refund mentioned, and yes, they would recommend.
Decent for the price, does the job, nothing fancy, medium positive score, defects not mentioned. So, again, Jev's performing accurately here. Terrible. Arrive broken. Unhelpful support. Want to refund. Strongly negative. Low score. High defect. Refund requested. So Jev is performing well when it's performing as expected, to rate these product reviews. I'm gonna try something a bit more, detailed in a moment's time.
And lastly, this fifth example is, based around customer churn. So we have some energy customers here, some IDs of those customers, the status of those customers, their payment types, their fuel types, and their balance. And we've... Maia is set up here to work out whether or not they're an engaged customer, what segmentation are they, and do they have any smart technology, based on the data that we have available. And then as a result of that, it's given us some low, medium, medium, high, and very high signals for customer churn analysis. So again, very easy to integrate out to Jev for these sorts of purposes.
Okay. So now we've got a sort of firm understanding of the three different, types of questions that we can leverage Jev for. So the choice, the score, and the null. Let's go ahead and actually, build a full sort of end to end pipeline that not only takes in real data, through Maia's ingestion capabilities, models it, gets it ready for downstream consumption, but then also, leverages the Jev model.
So to do this, I'm gonna come over here to Mission Control, which is where I sort of manage all of my tasks. I'm gonna come in here to, the project that I'm working in, and I've got some example branches in here as well. In the back to my Git repository, I'm gonna go into my own environment, but again, you could go into dev test or prod. I'm gonna give Maia a prompt.
So I, you know, I want to create a end to end, claims processing pipeline that leverages the Jev logic. You may see some other shared pipelines and skills inside of this project about how to build Jev integrated pipelines. I've got an S3 bucket with a variety of different data types, emails, JSON, text, and PDFs. I want you to leverage AWS's Textract components as well as the S3 load components to bring all of that data in, get it cleaned and ready in a bronze layer, and then in the silver layer, leverage Jev to create the sort of claims processing logic.
I want this to be metadata driven, and not just, a single feature build so that the logic can scale across other claims that we might see in the future and other sort of data source types that we might see in the future as well. Once you finish building, test all of the pipelines using the testing framework, document all of this and consult with me as you need. I'll give you the S3 bucket in a second.
So I'm gonna give Maia, a kind of balanced permission here to act autonomously on this. You can give it sort of full access but I kind of wanna be asked around, any sort of queries that it might need. I'm also gonna put it into plan mode, so I'm gonna give it a bit more time to think about how it wants to do this build. And this is the S3 bucket in question. So I've got some claims data in here. I'm gonna just copy that link, give that to Maia as well.
Here is that S3 link And it's gonna start leveraging the component palettes that we have inside of Maia's foundation to start understanding how it could build this out. So my prompt goes in here. You can see it's already invoked that Jev AI skill that I had Maia build earlier to work out how to take advantage of these new models from TypeSafe.
And, it's then also gonna go ahead and activate some built in Maia skills, so things like the shared pipelines functionality which allows us to create that reusable logic that we did, and also understands the parameters of how it can call and put those variables into that pipeline as and when it's needed, as well as the sort of testing, regression testing pipelines, the ability to create synthetic data to run through an example pipeline so that Maia can validate their business process, validate the output, and perform that verification check as it builds through the development life cycle.
So, you know, it's designed to be the autonomous data engineer that can take advantage of all of these capabilities inside of the Maia Foundation platform.
Featuring
You might also like
Our latest resources
Data management



