Text-to-image models offer unprecedented freedom to guide creation through natural language. Yet, it is unclear how such freedom can be exercised to generate images of specific unique concepts, modify their appearance, or compose them in new roles and novel scenes. In other words, we ask: how can we use language-guided models to turn our cat into a painting, or imagine a new product based on our favorite toy?

Here we present a simple approach that allows such creative freedom. Using only 3-5 images of a user-provided concept, like an object or a style, we learn to represent it through new "words" in the embedding space of a frozen text-to-image model. These "words" can be composed into natural language sentences, guiding personalized creation in an intuitive way.

Notably, we find evidence that a single word embedding is sufficient for capturing unique and varied concepts.

Bio: Daniel Cohen-Or is a professor in the School of Computer Science. He received his B.Sc. cum laude in both mathematics and computer science (1985), andM.Sc. cum laude in computer science (1986) from Ben-Gurion University, and Ph.D. from the Department of Computer Science (1991) at State University of New York at Stony Brook. He was sitting on the editorial board of a number of international journals, and a member of many the program committees of several international conferences.

He was the recipient of the Eurographics Outstanding Technical Contributions Award in 2005. In 2013 he received The People’s Republic of China Friendship Award. In 2015 he has been named a Thomson Reuters Highly Cited Researcherץ He received the ACM SIGGRAPH Computer Graphics Achievement Award in 2018. In 2019 he won The Kadar Family Award for Outstanding Research. In 2020 he received The Eurographics Distinguished Career Award.

His research interests are in computer graphics, in particular, synthesis, processing and modeling techniques

 
About the EuroCC-2 project

EuroCC 2 will work to identify and address the skills gaps in the European High Performance Computing (HPC) ecosystem and coordinate cooperation across Europe to ensure a consistent skills base.

The role of EuroCC 2 is to establish and run a network of more than 30 NCCs across the EuroHPC Participating States. The NCCs act as single points of access in each country between stakeholders and national and EuroHPC systems. They operate on a regional and national level to liaise with local communities, in particular SMEs, map HPC competencies and facilitate access to European HPC resources for users from the private and public sector.

EuroCC 2 delivers training, interacts with industry, develops competence mapping and communication materials and activities, and supports the adoption of HPC services in other related fields, such as quantum computing, artificial intelligence (AI), high performance data analytics (HPDA) to expand the HPC user base.

This project has received funding from the European High-Performance Computing Joint Undertaking (JU) under grant agreement No 101101903. The JU receives support from the European Union’s programme Digital Europe and Germany, Bulgaria, Austria, Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, Greece, Hungary, Ireland, Italy, Lithuania, Latvia, Poland, Portugal, Romania, Slovenia, Spain, Sweden, France, Netherlands, Belgium, Luxembourg , Slovakia, Norway, Switzerland, Turkey, the Republic of North Macedonia, Iceland, Montenegro and Serbia.