Generating latent representation of my own images

[ofirkris]

Hi, thanks for sharing this project.
I'm trying to create new images \ change Style of my own images with the FFHQ pre-trained model.
I've expected to get images as output, but instead I see .npy (numpy) file in "latent_representations" folder

What I tried:
'python align_images.py raw_images/ aligned_images/
python encode_images.py aligned_images/ generated_images/ latent_representations/'

What am I doing wrong?

[rolux]

Images should be in the generated_images/ directory.

[ofirkris]

I'm trying to achieve "Aging" of an image as I've done in the past using this encoder: https://github.com/pbaylies/stylegan-encoder (Fork for Puzer)

Latent directions - https://github.com/Puzer/stylegan-encoder/blob/master/Play_with_latent_directions.ipynb
Age - https://github.com/Puzer/stylegan-encoder/blob/master/ffhq_dataset/latent_directions/age.npy

When I run on a sample image the result looks nearly identical (Both images are attached seed6604_01.png is the result after inference)

Can you kindly show an inference example with age transform?

[rolux]

@ofirkris: Could it be that you're using StyleGAN V1 directions in Stylegan V2?

In that case, you'll need new ones.

[ofirkris]

Thanks!
So in order to do 'Age' inference, I need to load age.npy to latent_representations/?
Can you show the commands needed to run this type of inference?

[rolux]

https://gist.github.com/rolux/8073400130c91121561e3dc7cfd5421f

[ofirkris]

Thanks!
And to my last question - I get the same output as the above example - how do I control/change the input image to use on my own?

[rolux]

In the gist I linked, replace

z = np.random.RandomState(5616).randn(1, 512)
w = Gs.components.mapping.run(z, None)
w = w_avg + (w - w_avg) * truncation_psi

with

w = np.load('path/to/my/latent_representations/filename.npy')

[ofirkris]

For some reason it didn't run

I have Replaced this:
import numpy as np import PIL.Image w = np.load('age.npy') v_age = np.load('age.npy')

with:
w = np.load('latent_representations/seed6602_01.npy') v_age = np.load('age.npy')

Got this error:
`
Traceback (most recent call last):
File "/root/.virtualenvs/UGATIT/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1356, in _do_call
return fn(*args)
File "/root/.virtualenvs/UGATIT/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1341, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "/root/.virtualenvs/UGATIT/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1429, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: 2 root error(s) found.
(0) Invalid argument: In[0] is not a matrix. Instead it has shape [4]
[[{{node G_synthesis_1/_Run/G_synthesis/8x8/ToRGB/MatMul}}]]
[[G_synthesis_1/_Run/saturate_cast/_1453]]
(1) Invalid argument: In[0] is not a matrix. Instead it has shape [4]
[[{{node G_synthesis_1/_Run/G_synthesis/8x8/ToRGB/MatMul}}]]
0 successful operations.
0 derived errors ignored.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "age.py", line 44, in
image = Gs.components.synthesis.run(w_age, **Gs_syn_kwargs)[0]
File "/root/filters/stylegan2encoder/dnnlib/tflib/network.py", line 449, in run
mb_out = tf.get_default_session().run(out_expr, dict(zip(in_expr, mb_in)))
File "/root/.virtualenvs/UGATIT/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 950, in run
run_metadata_ptr)
File "/root/.virtualenvs/UGATIT/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1173, in _run
feed_dict_tensor, options, run_metadata)
File "/root/.virtualenvs/UGATIT/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1350, in _do_run
run_metadata)
File "/root/.virtualenvs/UGATIT/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1370, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: 2 root error(s) found.
(0) Invalid argument: In[0] is not a matrix. Instead it has shape [4]
[[node G_synthesis_1/_Run/G_synthesis/8x8/ToRGB/MatMul (defined at :46) ]]
[[G_synthesis_1/_Run/saturate_cast/_1453]]
(1) Invalid argument: In[0] is not a matrix. Instead it has shape [4]
[[node G_synthesis_1/_Run/G_synthesis/8x8/ToRGB/MatMul (defined at :46) ]]
0 successful operations.
0 derived errors ignored.

Errors may have originated from an input operation.
Input Source operations connected to node G_synthesis_1/_Run/G_synthesis/8x8/ToRGB/MatMul:
G_synthesis_1/_Run/G_synthesis/8x8/ToRGB/strided_slice (defined at :481)
G_synthesis_1/_Run/G_synthesis/8x8/ToRGB/mul_1 (defined at :36)

Input Source operations connected to node G_synthesis_1/_Run/G_synthesis/8x8/ToRGB/MatMul:
G_synthesis_1/_Run/G_synthesis/8x8/ToRGB/strided_slice (defined at :481)
G_synthesis_1/_Run/G_synthesis/8x8/ToRGB/mul_1 (defined at :36)

Original stack trace for 'G_synthesis_1/_Run/G_synthesis/8x8/ToRGB/MatMul':
File "age.py", line 44, in
image = Gs.components.synthesis.run(w_age, **Gs_syn_kwargs)[0]
File "/root/filters/stylegan2encoder/dnnlib/tflib/network.py", line 424, in run
out_gpu = net_gpu.get_output_for(*in_gpu, return_as_list=True, **dynamic_kwargs)
File "/root/filters/stylegan2encoder/dnnlib/tflib/network.py", line 221, in get_output_for
out_expr = self._build_func(*final_inputs, **build_kwargs)
File "", line 502, in G_synthesis_stylegan2
File "", line 481, in torgb
File "", line 98, in modulated_conv2d_layer
File "", line 46, in dense_layer
File "/root/.virtualenvs/UGATIT/lib/python3.6/site-packages/tensorflow/python/util/dispatch.py", line 180, in wrapper
return target(*args, **kwargs)
File "/root/.virtualenvs/UGATIT/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py", line 2647, in matmul
a, b, transpose_a=transpose_a, transpose_b=transpose_b, name=name)
File "/root/.virtualenvs/UGATIT/lib/python3.6/site-packages/tensorflow/python/ops/gen_math_ops.py", line 5925, in mat_mul
name=name)
File "/root/.virtualenvs/UGATIT/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 788, in _apply_op_helper
op_def=op_def)
File "/root/.virtualenvs/UGATIT/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/root/.virtualenvs/UGATIT/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3616, in create_op
op_def=op_def)
File "/root/.virtualenvs/UGATIT/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2005, in init
self._traceback = tf_stack.extract_stack()
`

[rolux]

What about... I'm closing this one and you DM me on twitter, or send me an e-mail? See profile.

[shiyingZhang90]

Hi @rolux @ofirkris ， I get exactly the same error as Ofirkris, any solution you found in the end? Is it due to latent vector we generated from real images have shape issues? Can you share the solution?

[Vis5]

Hello, I'm wondering about this as well.
EDIT: see #18 . I posted my fix