Research Scientist , Revamping the Role of Modern Flash Storage in Accelerating Generative AI Pipelines

myScience

Key information

Publication date:25 November 2024
Workload:100%
Place of work:Zurich
Salary estimate from jobs.ch:Log in, to see estimate from jobs.ch

Job summary

Join IBM Zurich Research Laboratory as a Research Scientist! This role offers an exciting opportunity to innovate in generative AI.

Tasks

Revamp the role of modern flash storage in AI pipelines.
Explore how NVMe technology can enhance data-intensive workflows.
Conduct research on storage dependencies in generative AI processes.

Skills

Ph.D. in computer science or electrical engineering required.
Proficiency in Python and C++ with strong coding skills.
Excellent communication and problem-solving abilities.

Is this helpful?

Research Scientist , Revamping the Role of Modern Flash Storage in Accelerating Generative AI Pipelines

IBM Zurich Research Laboratory

Workplace Zurich, Zurich region, Switzerland Category

Computer Science

Position

Senior Scientist / Postdoc

Published 21 November 2024

Research Scientist

Revamping the Role of Modern Flash Storage in Accelerating Generative AI Pipelines

Ref. 2024_031

Position description

Generative AI (GenAI) pipelines, especially using the RAG pattern, have attracted a significant amount of community attention due to their widespread use, effectiveness, and potential applicability in a variety of domains. In an ideal case, all data/metadata used in these workflows will be contained in memory. However, DRAM technology faces challenges from multiple fronts as it is not scaling, has a high cost ($/GB), and is energy inefficient. As the performance of Flash-based NVMe devices keeps increasing, the key research question that we are interested in is: how can modern Flash memory and NVMe hardware/software stacks help run data-intensive GenAI pipelines (focus on RAG, i.e. information retrieval and LLM inference)?

In this context, we aim to answer the following research questions:

What dependencies do RAG pipelines have on storage and how is storage accessed in RAG pipelines?

How can Vector DBs benefit from using high-performance NVMe flash arrays that can support millions of small I/O operations per seconds?

How can flash memory help accelerate the inference of LLMs?

Necessary qualifications

The candidate has obtained a Ph.D. degree in electrical engineering or computer science with a focus on (graph) data mining, (graph) databases, data storage, or (generative) AI in the past 5 years.

Excellent coding skills, in Python and C++, familiarity with Linux environments and software development tools (git/GitHub, IDEs, gcc, virtual machine and containers, etc.).

High level of creativity and outstanding problem-solving ability.

Excellent oral and written English and excellent presentation skills.

Strong academic record and experience with publishing and presenting at international conferences.

Strong interpersonal skills and excellent written and verbal communication.

Preferred Qualifications

Background and experience in systems architecture and hardware design.

Experience with systems programming and internals (kernel, memory management, hardware, CPU)

Experience with data storage and NVMe storage internals and specification.

Diversity

IBM is committed to diversity at the workplace. With us you will find an open, multicultural environment. Excellent flexible working arrangements enable all genders to strike the desired balance between their professional development and their personal lives.

How to apply

If you are interested in this position, please submit your application here.

More information

In your application, please refer to myScience.ch and referenceJobID65889.

Categories:

Public Admin. / Education / Social, Science / Research

Research Scientist , Revamping the Role of Modern Flash Storage in Accelerating Generative AI Pipelines

Key information

Tasks

Skills

Research Scientist , Revamping the Role of Modern Flash Storage in Accelerating Generative AI Pipelines

Revamping the Role of Modern Flash Storage in Accelerating Generative AI Pipelines

Categories:

Salary Check