How to retrieval using ComoDiff?

[develop-productivity]

Thanks you for porpose this excellent work！ But when I read the paper, I was very confused.
When we need to complete the retrieval task, I don't know how to complete the retrieval through the result generated by denoising. However, the demo of this warehouse only calculates the features of the image, but does not complete the retrieval, so how are the indicators of the paper calculated?
Can you help me? Thanks a lot.

[geonm]

Hi there.

CompoDiff/demo_search.py

Line 119 in 50e5ff6

sampled_image_features = model_dict['compodiff'].sample(image_cond, text_cond, negative_text_cond, mask, timesteps=step, cond_scale=cfg_scale, num_samples_per_batch=2)

In the above line, you can get the composed image features.

CompoDiff/demo_search.py

Lines 133 to 147 in 50e5ff6

	output = ''
	top1_list = []
	search_start = time.time()
	for name, features in do_list:
	results = client.query(embedding_input=features[0].tolist())[:15]
	output += f'<details open><summary>{name} outputs</summary>\n\n'
	for idx, result in enumerate(results):
	image_url = result['url']
	if idx == 0:
	top1_list.append(f'{image_url}')
	output += f'\n'
	output += '\n</details>\n\n'
	search_end = time.time()

and here you can retrieve top-K image urls using the composed image features.

Can you specify the meaning of the indicators of the paper?

[develop-productivity]

Hi there.

CompoDiff/demo_search.py

Line 119 in 50e5ff6

sampled_image_features = model_dict['compodiff'].sample(image_cond, text_cond, negative_text_cond, mask, timesteps=step, cond_scale=cfg_scale, num_samples_per_batch=2)

In the above line, you can get the composed image features.

CompoDiff/demo_search.py

Lines 133 to 147 in 50e5ff6

	output = ''
	top1_list = []
	search_start = time.time()
	for name, features in do_list:
	results = client.query(embedding_input=features[0].tolist())[:15]
	output += f'<details open><summary>{name} outputs</summary>\n\n'
	for idx, result in enumerate(results):
	image_url = result['url']
	if idx == 0:
	top1_list.append(f'{image_url}')
	output += f'\n'
	output += '\n</details>\n\n'
	search_end = time.time()

and here you can retrieve top-K image urls using the composed image features.
Can you specify the meaning of the indicators of the paper?

Thanks. I see how it works.
the indicators of the paper means recall@k。 I didn't describe it clearly. I'm sorry

[develop-productivity]

Hi there.

CompoDiff/demo_search.py

Line 119 in 50e5ff6

sampled_image_features = model_dict['compodiff'].sample(image_cond, text_cond, negative_text_cond, mask, timesteps=step, cond_scale=cfg_scale, num_samples_per_batch=2)

In the above line, you can get the composed image features.

CompoDiff/demo_search.py

Lines 133 to 147 in 50e5ff6

	output = ''
	top1_list = []
	search_start = time.time()
	for name, features in do_list:
	results = client.query(embedding_input=features[0].tolist())[:15]
	output += f'<details open><summary>{name} outputs</summary>\n\n'
	for idx, result in enumerate(results):
	image_url = result['url']
	if idx == 0:
	top1_list.append(f'{image_url}')
	output += f'\n'
	output += '\n</details>\n\n'
	search_end = time.time()

and here you can retrieve top-K image urls using the composed image features.
Can you specify the meaning of the indicators of the paper?

But I still don't understand how it works on CIRR or fashion-IQ datasets.
How to get the ground truth target image features using a diffusion model and caculate the recall metrics?
Thanks a lot.

[geonm]

Yes sure. This demo doesn't have the evaluate protocol for benchmark datasets.

We have currently only released the demo code to show the generalization performance of CompoDiff.

[develop-productivity]

Yes sure. This demo doesn't have the evaluate protocol for benchmark datasets.

We have currently only released the demo code to show the generalization performance of CompoDiff.

Do you plan to release the test code about protocol for benchmark datasets in the future? I'll be looking forward to it.