MBTA Info Hub

ABOUT THE PROJECT

Today callers to the MBTA’s paratransit service face long wait times and rerouting among call center operators. This chatbot enables paratransit service workers to answer disabled riders’ questions faster and more accurately, increasing rider satisfaction. Feedback was gathered through 4,500 survey responses and 8 in person interviews. The chatbot is in the process of being deployed.

Impact: The fully functional tool is being tested by Innovate Public School’s parent community in California.It is anticipated that the project will reduce call time, reduce call rerouting, increase accuracy, and increase rider satisfaction. The tool is currently in the integration phase, and they’ve indicated intent is to measure efficiency, resource allocation and user satisfaction. Key metrics will include user engagement, time saved, and the reduction of manual intervention needed to address common queries.

Partner: Massachusetts Bay Transportation Authority (MBTA)

Status: The Commonwealth of Massachusetts hired both students as interns. The agency plans to expand the chatbot’s usage within MBTA departments, enhancing its functionality, and to leverage generative AI in other areas such as document automation and processing large volumes of customer queries. 

PARTNERS

PROBLEM STATEMENT

Customers and other stakeholders of The RIDE often misunderstand or lack awareness of some RIDE policies and procedures, leading to repetitive calls about certain policies. Due to the structural fragmentation of the RIDE’s call centers (TRAC, Mobility Center, and The RIDE/MBTA), the staff members do not always have the answer to every customer inquiry. This puts pressure on the customer service teams and leads to excessive wait times and call rerouting. 

EXISTING SOLUTIONS

Currently, the only solution is for the customer service teams to take the call from customers and then reroute them or ask a supervisor if the question the customer asks is beyond their scope of knowledge. Information is mainly disseminated through a 16-page PDF, which itself does not have all information regarding The RIDE. A few Mobility Center staff members indicated that customers also do not read that entire packet. Other sources of information include a relatively user-friendly website that is not utilized fully due to customer preferences, as well as an FAQ document. For customers who prefer to call, there are four call centers, one of which only routes calls to the other three.

SOLUTIONS REQUIREMENTS

Requirements, Challenges, and Needs:
The target audience needs a reliable, fast source of information that they can use during a call to quickly access requisite information they need. For written inquiries, the users may also benefit from assistance with rapidly drafting written responses for emails or letters. Generative AI applications can span from creative and intelligent to very precise; this product should lean more towards the precise side. The accuracy of the tool is by far the most important requirement and extensive work must be put into ensuring that the tool does not deviate from its intended purpose or instructions.

Additionally, the set of information that the product must provide should encompass all RIDE-related information across both the Mobility Center, TRAC, and general RIDE procedures. An additional component may be to include information on disability services outside of the RIDE, such as the travel training provided by the Mobility Center.

Target Users:
The initial target audience for this product is the internal call-takers at the Mobility Center and TRAC, with the general MBTA support line as another possible audience. Staff members of each call center sometimes do not know about the policies handled at the other call centers, so when they receive calls that would pertain to another call center, they don’t know how to answer that question. This product is intended to bridge the knowledge gap between the call centers, providing a more seamless experience for callers who no longer need to be rerouted or wait for the receptionist to ask their supervisor for help.

As the product develops and improves, however, the audience may expand out to the MBTA support team (this team stands to benefit the most) and possibly general customers after that. A significant amount of time and care is necessary to ensure the reliability of the product, so segmenting the roll-out in this manner is a strong way to manage that requirement.

PRODUCT DESCRIPTION

We will build a generative AI chatbot that will allow customer support staff to easily access information about the RIDE, RIDE Flex, and potentially other disability and age-related services like travel training or senior discounts. This will allow call-takers to answer questions more quickly, especially if they are new or are being asked a question outside their domain of expertise (i.e. RIDE Flex questions to Mobility Center). This product is intended to have a flexible architecture that allows for potential deployment to customers themselves. However, we believe that it is best suited for initial release among customer service representatives first, and then expanding from there as needed. Finally, as calls are already labeled and logged in the current system, this tool can potentially support that functionality as well.

RISK & MITIGATION STRATEGIES

The following is a list of identified risks and the process used to mitigate each one. 

1. Accuracy of Information
   1. Careful and thorough prompting of the language model to constrain its outputs only to a narrow set of topics that strictly relate to The RIDE. 
2. Data Privacy and Security
   1. The product will run securely on AWS infrastructure and will not collect or use any user data for training purposes.
3. Reliability and Accountability
   1. Customer service representatives can validate and verify the accuracy of the information generated by the generative AI tool before it is relayed out to any customers. 

SUCCESS METRICS

Call Resolution Efficiency: Monitoring the time elapsed for calls as well as number of transfers. If customer service staff are able to access information outside their domain faster than they currently do, then the expectation is that some calls should resolve faster than before.

Application and Usage Rates: Observing any variations in the application rates to The RIDE service and the usage frequency of the service by the existing and new users.

User Satisfaction and Relevance: Measuring user satisfaction through surveys and feedback mechanisms to determine if the information provided post-implementation has become more accessible and relevant to the users’ needs. 

Call Volume Reduction: Monitoring a reduction in calls to the internal RIDE support team, provided there is widespread implementation at the Mobility Center as well as the MBTA support team.Approach:
Amazon Web Services (AWS) provides a comprehensive suite of tools to develop generative AI applications. It has features that provide scalable infrastructure, cost-effective tiers, robust security, and seamless integration into a variety of other AWS services.   We will use documents from the RIDE in order to provide our AI tool with the knowledge necessary to answer queries and possibly even execute tasks. AWS Bedrock will provide the APIs necessary to serve the language models that will generate the requisite responses. A simple web page through AWS Amplify or a similar framework will act as the interface between the user (a customer support representative) and the policy data for the RIDE.

RESEARCH FINDINGS

Generative AI Investigations:
Our own experimentation demonstrates that generative AI has a remarkable ability to summarize large amounts of data and produce output in a desired format. Prototypes using The RIDE Access Guide (the 16-page PDF of RIDE policies) and generative AI models demonstrate a strong ability to answer many kinds of questions. Generative AI models can also produce output using simpler and more concise phrasing than the original documents, which is critical for a service that supports such a broad group of people. 

Feasibility and Potential Advantages:
An AI tool is fast and responsive – for anything a customer support staff may not know, an AI tool can instantly look up the correct answer and provide an easy-to-read response for a staff member to relay back to a customer. As some of the service staff currently refer to their supervisors or other team members for information they don’t have themselves, this product serves as a single point of reference, removing the step of needing to refer to another human being. This is expected to improve the efficiency of call-takers. Value Proposition and Distinctive Benefits:
The implementation of this generative AI product in customer service teams will serve as a proof-of-concept for wider AI deployment. Customer service teams will act as expert users of this product, as they will be able to validate and provide feedback on the product’s outputs. As the product is improved, the workflow for the call center representatives is expected to speed up and encompass a wider array of information for them to relay to customers. These continuous improvements will pave the way for eventual use in customer-facing contexts.