Refining AI-Generated Messaging with Prompt Engineering: Human Evaluation, Iterative Improvement, and Adversarial Privacy Testing on Employee and Company Data

This thesis analyzes large language model (LLM) outputs in the automated Linkedin messaging task for commercial purposes. The main goals are to generate meaningful messages using different prompting strategies and test the robustness of the models with respect to the sensitive data in input. We will test different prompting strategies such as normal prompting , double prompting and further refinement using feedback. Additionally we will test the models with respect to the adversarial prompt injection task. We will provide feedback from the generations to further improve and refine the prompts, and in the end we will make human evaluation of generations based on several important metrics.