When deploying Large Language Models (LLMs) in voice-dialogue applications, long-term interaction is the norm, thus requiring continuous input-output capabilities. However, current LLM implementations do not support continuous inputs, only continuous output. The goal is to design and validate a streaming LLM that can achieve continuous input and real-time continuous output.

1. Local LLM (of more than 1B parameters) processing using a single GPU.
2. Capable of accepting infinite or large input token limits (such as 128K tokens).
3. Inputs are in a continuous stream, and the number of bytes added each time is not fixed.
4. The implementation should immediately pass the input into the model (considering the actual situation of the voice scenario, a very small buffer, such as no more than 128 bytes, can be retained).
5. As soon as a specified terminator (i.e. ) appears in the input stream, the model should start outputting as soon as possible, and the output throughput should not be lower than that of the original LLM model.
6. After a terminator appears in the input stream, if there is new incremental input, it should be treated as the next request for processing, and the context before the terminator should be considered when processing the next request.
7. The maximum interval between terminators should be similar to that of the current popular LLMs, such as 16KB (about 4K tokens). If the continuous input exceeds the maximum interval but no terminator appears, the previous content can be ignored.

Data for the problem statement is available at AWS S3 bucket
AWS Bucket Name: ps1and2-coderhackx-2023
AWS Access Key ID: AKIAR5ZTUIZKIVFNM654
AWS Secret Access Key: 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

Under the requirements and conditions of Problem Statement #01, we need to use a single LLM model on a single GPU to concurrently process multiple LLM tasks. The goal is to simultaneously process multiple LLM tasks under given GPU resources and keep the response time for each task within an acceptable range.

Data for the problem statement is available at AWS S3 bucket
AWS Bucket Name: ps1and2-coderhackx-2023
AWS Access Key ID: AKIAR5ZTUIZKIVFNM654
AWS Secret Access Key: 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

Unable to Forecast Demand in Branch activity and optimize branch load for customers booking appointments.

Unable to recommend solutions to customers to eliminate or deliver through self-serve platforms.
Solution Required:
Maximize operational efficiency by analyzing data to improve staffing, resource allocation, and customer service at their physical branches

Benefits to clients (Banks)

• Optimize staffing
• Increase branch profitability
• Reduce Wait-times
• Cost savings

Expected Output from solution provider
• Recommended approach to solve the problem
• Solution architecture
• Proof of concept
• Auto-generated flat files or API that can be integrated with application

Synthetic data required

• Branch level history data
• Transactions history data

Data for the problem statement is available at AWS S3 bucket
AWS Bucket Name: ps-3-coderhackx-2023

AWS Access Key ID: AKIAR5ZTUIZKMPUCX2E5
AWS Secret Access Key: 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