Edb-Ipp Project Personalization and Long-Context Modeling in Large Language Models

**Job Description:** Rakuten Asia, in partnership with the Economic Development Board (**EDB**) through the Industrial Postgraduate Programme (**IPP**), is seeking new PhD students. We are looking for individuals with a robust understanding of deep learning, machine learning, and natural language processing to contribute to our innovative research projects. Essential requirements include proven hands-on expertise and strong engineering skillsets, specifically in the development and training of PyTorch models. **IPP Programme Benefits** Candidates successfully selected for this programme will receive full sponsorship for their postgraduate studies and will be hired by Rakuten Asia upon successful completion. **Collaboration Model** The collaboration will include joint supervision of PhD students, shared infrastructure access for large-scale experiments, and regular research exchange. Outputs will include publications, open-source prototypes, and scalable frameworks for personalized LLM deployment. **Project Outline** **Project Title: Personalization and Long-Context Modeling in Large Language Models** **Introduction**   We are entering a phase in the development of Large Language Models (LLMs), where personalization, long-context understanding, and continual interaction with users are becoming critical differentiators. This research initiative seeks to push the boundaries of user-adaptive, memory-augmented, and privacy-aware LLMs that can reason over long-term history while preserving efficiency and alignment. Our team brings experience in training and evaluating cutting-edge LLMs, and we invite academic collaborators to join us in shaping the next generation of user-centric AI systems. **Objectives** This collaboration aims to: * Advance foundational techniques for personalizing LLMs over long, evolving contexts. * Develop scalable methods for encoding and decoding extended user interaction history. * Benchmark and prototype systems that integrate memory, retrieval, and adaptation in real-world applications. * Train and support PhD-level talent through joint supervision and research internships. **Proposed Research Areas** We propose collaboration across the following topics, with openness to refining based on shared interests: * **Long-Context Representation and Compression** Explore architectures (e.g. retrieval-augmented, segment-aware transformers, state-space models) that can efficiently handle user histories spanning millions of tokens. * **Personalization without Fine-Tuning** Develop modular personalization techniques using adapters and user embeddings. Emphasize continual learning methods that avoid full-model retraining. * **Alignment and Safety for Personalized Models** Develop evaluation protocols and mitigation strategies to ensure personalized behavior remains aligned with safety constraints and social norms under extensive user adaptation. * **Efficient Infrastructure for Persistent Context** Design systems that support long-term memory and personalization with low-latency access to evolving user state. _Rakuten is an equal opportunities employer and welcomes applications regardless of sex, marital status, ethnic origin, sexual orientation, religious belief or age._