Name Your Colour For the Task: Artificially Discover Colour Naming via Colour Quantisation Transformer
Shenghan Su1
Lin Gu3,2*
Yue Yang1,4
Zenghui Zhang1
Tatsuya Harada2,3
1. Shanghai Jiao Tong University 
2. The University of Tokyo 
3. RIKEN AIP 
4. Shanghai AI Laboratory
[Paper]
[GitHub]
(a) The theoretical limit of efficiency for colour naming (black curve) and cases of the WCS probability map of human colour language copied from Zaslavsky et al. (b) The colour size-accuracy curve and the WCS probability maps generated by our CQFormer. (c) The colour naming stimulus grid used in the WCS. (d) The three-term WCS probability map of CQFormer after embedding 1978 Nafaanra three-colour system ((light (`fiNge'), dark (`wOO'), and warm or red-like (`nyiE')) into the latent representation. (e) The four-term WCS probability map of CQFormer evolved from (d). The evolved fourth colour, yellow-green, is consistent with the prediction of basic colour term theory (Brent Berlin and Paul Kay).

Abstract

The long-standing theory that a colour-naming system evolves under dual pressure of efficient communication and perceptual mechanism is supported by more and more linguistic studies, including analysing four decades of diachronic data from the Nafaanra language. This inspires us to explore whether machine learning could evolve and discover a similar colour-naming system via optimising the communication efficiency represented by high-level recognition performance. Here, we propose a novel colour quantisation transformer, CQFormer, that quantises colour space while maintaining the accuracy of machine recognition on the quantised images. Given an RGB image, Annotation Branch maps it into an index map before generating the quantised image with a colour palette; meanwhile the Palette Branch utilises a key-point detection way to find proper colours in the palette among the whole colour space. By interacting with colour annotation, CQFormer is able to balance both the machine vision accuracy and colour perceptual structure such as distinct and stable colour distribution for discovered colour system. Very interestingly, we even observe the consistent evolution pattern between our artificial colour system and basic colour terms across human languages. Besides, our colour quantisation method also offers an efficient quantisation method that effectively compresses the image storage while maintaining high performance in high-level recognition tasks such as classification and detection. Extensive experiments demonstrate the superior performance of our method with extremely low bit-rate colours, showing potential to integrate into quantisation network to quantities from image to network activation.


Poster


[Slides]


 [GitHub]


Paper and Supplementary Material

Shenghan Su, Lin Gu, Yue Yang, Zenghui Zhang, Tatsuya Harada
Name Your Colour For the Task: Artificially Discover Colour Naming via Colour Quantisation Transformer.
Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023.



[Bibtex]


Acknowledgements

This template was originally made by Phillip Isola and Richard Zhang for a colorful ECCV project; the code can be found here.