How to Fix Shape Mismatch in TensorFlow when attempting to create a model from a trained data set

I am working with machine learning project using TensorFlow and Keras to process audio data. The project is designed to handle inputs with multiple channels, but there’s an issue during the training phase where the input shape does not match the expected model input shape.

In the github repo you can find model.py, I think that is supposed to reduce the channel dimension of audio data from 3 to 1 using a TimeDistributed layer wrapped around a Conv2D layer. However, during training, I’m encountering a shape mismatch error:

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
<code>TypeError: `generator` yielded an element of shape (32, 2, 15, 80, 3) where an element of shape (None, 2, 15, 80, 1) was expected.
</code>
<code>TypeError: `generator` yielded an element of shape (32, 2, 15, 80, 3) where an element of shape (None, 2, 15, 80, 1) was expected. </code>
TypeError: `generator` yielded an element of shape (32, 2, 15, 80, 3) where an element of shape (None, 2, 15, 80, 1) was expected.

The github repo is here https://github.com/cpuguy96/StepCOVNet

I used that repo to prep the training data from the raw data and went to create the model which gives the mismatch.

The prints before seem to suggest some shapes some of the time are correct?

I’m just not sure where to go from here. My goal is to create the model and use it for inferencing a novel song into stepmania step data.

I’m not sure if the repo is just in a bad unworking state or if its user error on my part.

I tend to learn by doing and thought this would be a good project to start in.

Exact command and output.

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
<code>(StepNet) nonlin@Nonlin:/mnt/c/Users/gerfe/StepCOVNet$ python train.py -i /home/nonlin/miniconda3/envs/StepNet/TrainingData -o /home/nonlin/miniconda3/envs/StepNet/Model -d 1 --name "MyModel"
2024-05-11 10:00:27.499059: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-05-11 10:00:27.499132: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-05-11 10:00:27.516496: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-05-11 10:00:27.564170: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-05-11 10:00:28.449546: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
/home/nonlin/miniconda3/envs/StepNet/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
warnings.warn(
2024-05-11 10:00:30.166859: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:887] could not open file to read NUMA node: /sys/bus/pci/devices/0000:0b:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-05-11 10:00:30.508089: W tensorflow/core/common_runtime/gpu/gpu_device.cc:2256] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
WARNING:root:Failed to set memory growth for GPU.
Traceback (most recent call last):
File "/mnt/c/Users/gerfe/StepCOVNet/stepcovnet/tf_config.py", line 10, in <module>
tf.config.list_physical_devices("GPU")[0], enable=True
IndexError: list index out of range
WARNING:tensorflow:Mixed precision compatibility check (mixed_float16): WARNING
The dtype policy mixed_float16 may run slowly because this machine does not have a GPU. Only Nvidia GPUs with compute capability of at least 7.0 run quickly with mixed_float16.
If you will use compatible GPU(s) not attached to this host, e.g. by running a multi-worker model, you can ignore this warning. This message will only be logged once
WARNING:tensorflow:Mixed precision compatibility check (mixed_float16): WARNING
The dtype policy mixed_float16 may run slowly because this machine does not have a GPU. Only Nvidia GPUs with compute capability of at least 7.0 run quickly with mixed_float16.
If you will use compatible GPU(s) not attached to this host, e.g. by running a multi-worker model, you can ignore this warning. This message will only be logged once
/home/nonlin/miniconda3/envs/StepNet/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
warnings.warn(
2024-05-11 10:02:19.167084: W external/local_tsl/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 154389504 exceeds 10% of free system memory.
2024-05-11 10:02:19.202167: W external/local_tsl/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 154389504 exceeds 10% of free system memory.
2024-05-11 10:02:19.232825: W external/local_tsl/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 154389504 exceeds 10% of free system memory.
2024-05-11 10:02:20.980241: W external/local_tsl/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 154389504 exceeds 10% of free system memory.
All PyTorch model weights were used when initializing TFGPT2Model.
All the weights of TFGPT2Model were initialized from the PyTorch model.
If your task is similar to the task the model of the checkpoint was trained on, you can already use TFGPT2Model for predictions without further training.
Model: "StepCOVNet"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
audio_input (InputLayer) [(None, 2, 15, 80, 1)] 0 []
time_distributed (TimeDist (None, 2, 15, 80, 1) 2 ['audio_input[0][0]']
ributed)
arrow_input (InputLayer) [(None, None)] 0 []
arrow_mask (InputLayer) [(None, None)] 0 []
batch_normalization (Batch (None, 2, 15, 80, 1) 4 ['time_distributed[0][0]']
Normalization)
tfgpt2_model (TFGPT2Model) TFBaseModelOutputWithPastA 1244398 ['arrow_input[0][0]',
ndCrossAttentions(last_hid 08 'arrow_mask[0][0]']
den_state=(None, None, 768
),
past_key_values=((2, None
, 12, None, 64),
(2, None, 12, None, 64),
(2, None, 12, None, 64),
(2, None, 12, None, 64),
(2, None, 12, None, 64),
(2, None, 12, None, 64),
(2, None, 12, None, 64),
(2, None, 12, None, 64),
(2, None, 12, None, 64),
(2, None, 12, None, 64),
(2, None, 12, None, 64),
(2, None, 12, None, 64)),
hidden_states=None, atten
tions=None, cross_attentio
ns=None)
VGGish (Functional) (None, 2, 512) 4499712 ['batch_normalization[0][0]']
global_max_pooling1d (Glob (None, 768) 0 ['tfgpt2_model[0][0]']
alMaxPooling1D)
bidirectional (Bidirection (None, 256) 656384 ['VGGish[0][0]']
al)
concatenate (Concatenate) (None, 1024) 0 ['global_max_pooling1d[0][0]',
'bidirectional[0][0]']
dense (Dense) (None, 256) 262400 ['concatenate[0][0]']
batch_normalization_1 (Bat (None, 256) 1024 ['dense[0][0]']
chNormalization)
activation (Activation) (None, 256) 0 ['batch_normalization_1[0][0]'
]
dropout_37 (Dropout) (None, 256) 0 ['activation[0][0]']
onehot_encoded_arrows (Den (None, 256) 65792 ['dropout_37[0][0]']
se)
==================================================================================================
Total params: 129925126 (495.63 MB)
Trainable params: 40370436 (154.00 MB)
Non-trainable params: 89554690 (341.62 MB)
__________________________________________________________________________________________________
Saving model metadata at /home/nonlin/miniconda3/envs/StepNet/Model/MyModel
Training on 1975957 samples (156 songs) and validating on 231660 samples (18 songs)
Starting training...
Epoch 1/15
2024-05-11 10:02:28.230065: W external/local_tsl/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 154389504 exceeds 10% of free system memory.
2024-05-11 10:02:36.035559: W tensorflow/core/framework/op_kernel.cc:1827] INVALID_ARGUMENT: TypeError: `generator` yielded an element of shape (32, 2, 15, 80, 3) where an element of shape (None, 2, 15, 80, 1) was expected.
Traceback (most recent call last):
File "/home/nonlin/miniconda3/envs/StepNet/lib/python3.10/site-packages/tensorflow/python/ops/script_ops.py", line 270, in __call__
ret = func(*args)
File "/home/nonlin/miniconda3/envs/StepNet/lib/python3.10/site-packages/tensorflow/python/autograph/impl/api.py", line 643, in wrapper
return func(*args, **kwargs)
File "/home/nonlin/miniconda3/envs/StepNet/lib/python3.10/site-packages/tensorflow/python/data/ops/from_generator_op.py", line 235, in generator_py_func
raise TypeError(
TypeError: `generator` yielded an element of shape (32, 2, 15, 80, 3) where an element of shape (None, 2, 15, 80, 1) was expected.
Traceback (most recent call last):
File "/mnt/c/Users/gerfe/StepCOVNet/train.py", line 166, in <module>
train(
File "/mnt/c/Users/gerfe/StepCOVNet/train.py", line 115, in train
run_training(
File "/mnt/c/Users/gerfe/StepCOVNet/train.py", line 61, in run_training
executor.TrainingExecutor(stepcovnet_model=stepcovnet_model).execute(
File "/mnt/c/Users/gerfe/StepCOVNet/stepcovnet/executor.py", line 111, in execute
history = self.train(input_data, self.get_training_callbacks(hyperparameters))
File "/mnt/c/Users/gerfe/StepCOVNet/stepcovnet/executor.py", line 191, in train
history = self.stepcovnet_model.model.fit(
File "/home/nonlin/miniconda3/envs/StepNet/lib/python3.10/site-packages/keras/src/utils/traceback_utils.py", line 70, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/home/nonlin/miniconda3/envs/StepNet/lib/python3.10/site-packages/tensorflow/python/eager/execute.py", line 53, in quick_execute
tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
tensorflow.python.framework.errors_impl.InvalidArgumentError: Graph execution error:
Detected at node PyFunc defined at (most recent call last):
<stack traces unavailable>
TypeError: `generator` yielded an element of shape (32, 2, 15, 80, 3) where an element of shape (None, 2, 15, 80, 1) was expected.
Traceback (most recent call last):
File "/home/nonlin/miniconda3/envs/StepNet/lib/python3.10/site-packages/tensorflow/python/ops/script_ops.py", line 270, in __call__
ret = func(*args)
File "/home/nonlin/miniconda3/envs/StepNet/lib/python3.10/site-packages/tensorflow/python/autograph/impl/api.py", line 643, in wrapper
return func(*args, **kwargs)
File "/home/nonlin/miniconda3/envs/StepNet/lib/python3.10/site-packages/tensorflow/python/data/ops/from_generator_op.py", line 235, in generator_py_func
raise TypeError(
TypeError: `generator` yielded an element of shape (32, 2, 15, 80, 3) where an element of shape (None, 2, 15, 80, 1) was expected.
[[{{node PyFunc}}]]
[[IteratorGetNext]] [Op:__inference_train_function_33834]
</code>
<code>(StepNet) nonlin@Nonlin:/mnt/c/Users/gerfe/StepCOVNet$ python train.py -i /home/nonlin/miniconda3/envs/StepNet/TrainingData -o /home/nonlin/miniconda3/envs/StepNet/Model -d 1 --name "MyModel" 2024-05-11 10:00:27.499059: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered 2024-05-11 10:00:27.499132: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered 2024-05-11 10:00:27.516496: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered 2024-05-11 10:00:27.564170: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations. To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags. 2024-05-11 10:00:28.449546: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT /home/nonlin/miniconda3/envs/StepNet/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. warnings.warn( 2024-05-11 10:00:30.166859: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:887] could not open file to read NUMA node: /sys/bus/pci/devices/0000:0b:00.0/numa_node Your kernel may have been built without NUMA support. 2024-05-11 10:00:30.508089: W tensorflow/core/common_runtime/gpu/gpu_device.cc:2256] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform. Skipping registering GPU devices... WARNING:root:Failed to set memory growth for GPU. Traceback (most recent call last): File "/mnt/c/Users/gerfe/StepCOVNet/stepcovnet/tf_config.py", line 10, in <module> tf.config.list_physical_devices("GPU")[0], enable=True IndexError: list index out of range WARNING:tensorflow:Mixed precision compatibility check (mixed_float16): WARNING The dtype policy mixed_float16 may run slowly because this machine does not have a GPU. Only Nvidia GPUs with compute capability of at least 7.0 run quickly with mixed_float16. If you will use compatible GPU(s) not attached to this host, e.g. by running a multi-worker model, you can ignore this warning. This message will only be logged once WARNING:tensorflow:Mixed precision compatibility check (mixed_float16): WARNING The dtype policy mixed_float16 may run slowly because this machine does not have a GPU. Only Nvidia GPUs with compute capability of at least 7.0 run quickly with mixed_float16. If you will use compatible GPU(s) not attached to this host, e.g. by running a multi-worker model, you can ignore this warning. This message will only be logged once /home/nonlin/miniconda3/envs/StepNet/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. warnings.warn( 2024-05-11 10:02:19.167084: W external/local_tsl/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 154389504 exceeds 10% of free system memory. 2024-05-11 10:02:19.202167: W external/local_tsl/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 154389504 exceeds 10% of free system memory. 2024-05-11 10:02:19.232825: W external/local_tsl/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 154389504 exceeds 10% of free system memory. 2024-05-11 10:02:20.980241: W external/local_tsl/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 154389504 exceeds 10% of free system memory. All PyTorch model weights were used when initializing TFGPT2Model. All the weights of TFGPT2Model were initialized from the PyTorch model. If your task is similar to the task the model of the checkpoint was trained on, you can already use TFGPT2Model for predictions without further training. Model: "StepCOVNet" __________________________________________________________________________________________________ Layer (type) Output Shape Param # Connected to ================================================================================================== audio_input (InputLayer) [(None, 2, 15, 80, 1)] 0 [] time_distributed (TimeDist (None, 2, 15, 80, 1) 2 ['audio_input[0][0]'] ributed) arrow_input (InputLayer) [(None, None)] 0 [] arrow_mask (InputLayer) [(None, None)] 0 [] batch_normalization (Batch (None, 2, 15, 80, 1) 4 ['time_distributed[0][0]'] Normalization) tfgpt2_model (TFGPT2Model) TFBaseModelOutputWithPastA 1244398 ['arrow_input[0][0]', ndCrossAttentions(last_hid 08 'arrow_mask[0][0]'] den_state=(None, None, 768 ), past_key_values=((2, None , 12, None, 64), (2, None, 12, None, 64), (2, None, 12, None, 64), (2, None, 12, None, 64), (2, None, 12, None, 64), (2, None, 12, None, 64), (2, None, 12, None, 64), (2, None, 12, None, 64), (2, None, 12, None, 64), (2, None, 12, None, 64), (2, None, 12, None, 64), (2, None, 12, None, 64)), hidden_states=None, atten tions=None, cross_attentio ns=None) VGGish (Functional) (None, 2, 512) 4499712 ['batch_normalization[0][0]'] global_max_pooling1d (Glob (None, 768) 0 ['tfgpt2_model[0][0]'] alMaxPooling1D) bidirectional (Bidirection (None, 256) 656384 ['VGGish[0][0]'] al) concatenate (Concatenate) (None, 1024) 0 ['global_max_pooling1d[0][0]', 'bidirectional[0][0]'] dense (Dense) (None, 256) 262400 ['concatenate[0][0]'] batch_normalization_1 (Bat (None, 256) 1024 ['dense[0][0]'] chNormalization) activation (Activation) (None, 256) 0 ['batch_normalization_1[0][0]' ] dropout_37 (Dropout) (None, 256) 0 ['activation[0][0]'] onehot_encoded_arrows (Den (None, 256) 65792 ['dropout_37[0][0]'] se) ================================================================================================== Total params: 129925126 (495.63 MB) Trainable params: 40370436 (154.00 MB) Non-trainable params: 89554690 (341.62 MB) __________________________________________________________________________________________________ Saving model metadata at /home/nonlin/miniconda3/envs/StepNet/Model/MyModel Training on 1975957 samples (156 songs) and validating on 231660 samples (18 songs) Starting training... Epoch 1/15 2024-05-11 10:02:28.230065: W external/local_tsl/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 154389504 exceeds 10% of free system memory. 2024-05-11 10:02:36.035559: W tensorflow/core/framework/op_kernel.cc:1827] INVALID_ARGUMENT: TypeError: `generator` yielded an element of shape (32, 2, 15, 80, 3) where an element of shape (None, 2, 15, 80, 1) was expected. Traceback (most recent call last): File "/home/nonlin/miniconda3/envs/StepNet/lib/python3.10/site-packages/tensorflow/python/ops/script_ops.py", line 270, in __call__ ret = func(*args) File "/home/nonlin/miniconda3/envs/StepNet/lib/python3.10/site-packages/tensorflow/python/autograph/impl/api.py", line 643, in wrapper return func(*args, **kwargs) File "/home/nonlin/miniconda3/envs/StepNet/lib/python3.10/site-packages/tensorflow/python/data/ops/from_generator_op.py", line 235, in generator_py_func raise TypeError( TypeError: `generator` yielded an element of shape (32, 2, 15, 80, 3) where an element of shape (None, 2, 15, 80, 1) was expected. Traceback (most recent call last): File "/mnt/c/Users/gerfe/StepCOVNet/train.py", line 166, in <module> train( File "/mnt/c/Users/gerfe/StepCOVNet/train.py", line 115, in train run_training( File "/mnt/c/Users/gerfe/StepCOVNet/train.py", line 61, in run_training executor.TrainingExecutor(stepcovnet_model=stepcovnet_model).execute( File "/mnt/c/Users/gerfe/StepCOVNet/stepcovnet/executor.py", line 111, in execute history = self.train(input_data, self.get_training_callbacks(hyperparameters)) File "/mnt/c/Users/gerfe/StepCOVNet/stepcovnet/executor.py", line 191, in train history = self.stepcovnet_model.model.fit( File "/home/nonlin/miniconda3/envs/StepNet/lib/python3.10/site-packages/keras/src/utils/traceback_utils.py", line 70, in error_handler raise e.with_traceback(filtered_tb) from None File "/home/nonlin/miniconda3/envs/StepNet/lib/python3.10/site-packages/tensorflow/python/eager/execute.py", line 53, in quick_execute tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name, tensorflow.python.framework.errors_impl.InvalidArgumentError: Graph execution error: Detected at node PyFunc defined at (most recent call last): <stack traces unavailable> TypeError: `generator` yielded an element of shape (32, 2, 15, 80, 3) where an element of shape (None, 2, 15, 80, 1) was expected. Traceback (most recent call last): File "/home/nonlin/miniconda3/envs/StepNet/lib/python3.10/site-packages/tensorflow/python/ops/script_ops.py", line 270, in __call__ ret = func(*args) File "/home/nonlin/miniconda3/envs/StepNet/lib/python3.10/site-packages/tensorflow/python/autograph/impl/api.py", line 643, in wrapper return func(*args, **kwargs) File "/home/nonlin/miniconda3/envs/StepNet/lib/python3.10/site-packages/tensorflow/python/data/ops/from_generator_op.py", line 235, in generator_py_func raise TypeError( TypeError: `generator` yielded an element of shape (32, 2, 15, 80, 3) where an element of shape (None, 2, 15, 80, 1) was expected. [[{{node PyFunc}}]] [[IteratorGetNext]] [Op:__inference_train_function_33834] </code>
(StepNet) nonlin@Nonlin:/mnt/c/Users/gerfe/StepCOVNet$ python train.py -i /home/nonlin/miniconda3/envs/StepNet/TrainingData -o /home/nonlin/miniconda3/envs/StepNet/Model -d 1 --name "MyModel"
2024-05-11 10:00:27.499059: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-05-11 10:00:27.499132: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-05-11 10:00:27.516496: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-05-11 10:00:27.564170: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-05-11 10:00:28.449546: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
/home/nonlin/miniconda3/envs/StepNet/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
  warnings.warn(
2024-05-11 10:00:30.166859: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:887] could not open file to read NUMA node: /sys/bus/pci/devices/0000:0b:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-05-11 10:00:30.508089: W tensorflow/core/common_runtime/gpu/gpu_device.cc:2256] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
WARNING:root:Failed to set memory growth for GPU.
Traceback (most recent call last):
  File "/mnt/c/Users/gerfe/StepCOVNet/stepcovnet/tf_config.py", line 10, in <module>
    tf.config.list_physical_devices("GPU")[0], enable=True
IndexError: list index out of range
WARNING:tensorflow:Mixed precision compatibility check (mixed_float16): WARNING
The dtype policy mixed_float16 may run slowly because this machine does not have a GPU. Only Nvidia GPUs with compute capability of at least 7.0 run quickly with mixed_float16.
If you will use compatible GPU(s) not attached to this host, e.g. by running a multi-worker model, you can ignore this warning. This message will only be logged once
WARNING:tensorflow:Mixed precision compatibility check (mixed_float16): WARNING
The dtype policy mixed_float16 may run slowly because this machine does not have a GPU. Only Nvidia GPUs with compute capability of at least 7.0 run quickly with mixed_float16.
If you will use compatible GPU(s) not attached to this host, e.g. by running a multi-worker model, you can ignore this warning. This message will only be logged once
/home/nonlin/miniconda3/envs/StepNet/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
  warnings.warn(
2024-05-11 10:02:19.167084: W external/local_tsl/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 154389504 exceeds 10% of free system memory.
2024-05-11 10:02:19.202167: W external/local_tsl/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 154389504 exceeds 10% of free system memory.
2024-05-11 10:02:19.232825: W external/local_tsl/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 154389504 exceeds 10% of free system memory.
2024-05-11 10:02:20.980241: W external/local_tsl/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 154389504 exceeds 10% of free system memory.
All PyTorch model weights were used when initializing TFGPT2Model.

All the weights of TFGPT2Model were initialized from the PyTorch model.
If your task is similar to the task the model of the checkpoint was trained on, you can already use TFGPT2Model for predictions without further training.
Model: "StepCOVNet"
__________________________________________________________________________________________________
 Layer (type)                Output Shape                 Param #   Connected to
==================================================================================================
 audio_input (InputLayer)    [(None, 2, 15, 80, 1)]       0         []

 time_distributed (TimeDist  (None, 2, 15, 80, 1)         2         ['audio_input[0][0]']
 ributed)

 arrow_input (InputLayer)    [(None, None)]               0         []

 arrow_mask (InputLayer)     [(None, None)]               0         []

 batch_normalization (Batch  (None, 2, 15, 80, 1)         4         ['time_distributed[0][0]']
 Normalization)

 tfgpt2_model (TFGPT2Model)  TFBaseModelOutputWithPastA   1244398   ['arrow_input[0][0]',
                             ndCrossAttentions(last_hid   08         'arrow_mask[0][0]']
                             den_state=(None, None, 768
                             ),
                              past_key_values=((2, None
                             , 12, None, 64),
                              (2, None, 12, None, 64),
                              (2, None, 12, None, 64),
                              (2, None, 12, None, 64),
                              (2, None, 12, None, 64),
                              (2, None, 12, None, 64),
                              (2, None, 12, None, 64),
                              (2, None, 12, None, 64),
                              (2, None, 12, None, 64),
                              (2, None, 12, None, 64),
                              (2, None, 12, None, 64),
                              (2, None, 12, None, 64)),
                              hidden_states=None, atten
                             tions=None, cross_attentio
                             ns=None)

 VGGish (Functional)         (None, 2, 512)               4499712   ['batch_normalization[0][0]']

 global_max_pooling1d (Glob  (None, 768)                  0         ['tfgpt2_model[0][0]']
 alMaxPooling1D)

 bidirectional (Bidirection  (None, 256)                  656384    ['VGGish[0][0]']
 al)

 concatenate (Concatenate)   (None, 1024)                 0         ['global_max_pooling1d[0][0]',
                                                                     'bidirectional[0][0]']

 dense (Dense)               (None, 256)                  262400    ['concatenate[0][0]']

 batch_normalization_1 (Bat  (None, 256)                  1024      ['dense[0][0]']
 chNormalization)

 activation (Activation)     (None, 256)                  0         ['batch_normalization_1[0][0]'
                                                                    ]

 dropout_37 (Dropout)        (None, 256)                  0         ['activation[0][0]']

 onehot_encoded_arrows (Den  (None, 256)                  65792     ['dropout_37[0][0]']
 se)

==================================================================================================
Total params: 129925126 (495.63 MB)
Trainable params: 40370436 (154.00 MB)
Non-trainable params: 89554690 (341.62 MB)
__________________________________________________________________________________________________
Saving model metadata at /home/nonlin/miniconda3/envs/StepNet/Model/MyModel
Training on 1975957 samples (156 songs) and validating on 231660 samples (18 songs)

Starting training...
Epoch 1/15
2024-05-11 10:02:28.230065: W external/local_tsl/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 154389504 exceeds 10% of free system memory.
2024-05-11 10:02:36.035559: W tensorflow/core/framework/op_kernel.cc:1827] INVALID_ARGUMENT: TypeError: `generator` yielded an element of shape (32, 2, 15, 80, 3) where an element of shape (None, 2, 15, 80, 1) was expected.
Traceback (most recent call last):

  File "/home/nonlin/miniconda3/envs/StepNet/lib/python3.10/site-packages/tensorflow/python/ops/script_ops.py", line 270, in __call__
    ret = func(*args)

  File "/home/nonlin/miniconda3/envs/StepNet/lib/python3.10/site-packages/tensorflow/python/autograph/impl/api.py", line 643, in wrapper
    return func(*args, **kwargs)

  File "/home/nonlin/miniconda3/envs/StepNet/lib/python3.10/site-packages/tensorflow/python/data/ops/from_generator_op.py", line 235, in generator_py_func
    raise TypeError(

TypeError: `generator` yielded an element of shape (32, 2, 15, 80, 3) where an element of shape (None, 2, 15, 80, 1) was expected.


Traceback (most recent call last):
  File "/mnt/c/Users/gerfe/StepCOVNet/train.py", line 166, in <module>
    train(
  File "/mnt/c/Users/gerfe/StepCOVNet/train.py", line 115, in train
    run_training(
  File "/mnt/c/Users/gerfe/StepCOVNet/train.py", line 61, in run_training
    executor.TrainingExecutor(stepcovnet_model=stepcovnet_model).execute(
  File "/mnt/c/Users/gerfe/StepCOVNet/stepcovnet/executor.py", line 111, in execute
    history = self.train(input_data, self.get_training_callbacks(hyperparameters))
  File "/mnt/c/Users/gerfe/StepCOVNet/stepcovnet/executor.py", line 191, in train
    history = self.stepcovnet_model.model.fit(
  File "/home/nonlin/miniconda3/envs/StepNet/lib/python3.10/site-packages/keras/src/utils/traceback_utils.py", line 70, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/home/nonlin/miniconda3/envs/StepNet/lib/python3.10/site-packages/tensorflow/python/eager/execute.py", line 53, in quick_execute
    tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
tensorflow.python.framework.errors_impl.InvalidArgumentError: Graph execution error:

Detected at node PyFunc defined at (most recent call last):
<stack traces unavailable>
TypeError: `generator` yielded an element of shape (32, 2, 15, 80, 3) where an element of shape (None, 2, 15, 80, 1) was expected.
Traceback (most recent call last):

  File "/home/nonlin/miniconda3/envs/StepNet/lib/python3.10/site-packages/tensorflow/python/ops/script_ops.py", line 270, in __call__
    ret = func(*args)

  File "/home/nonlin/miniconda3/envs/StepNet/lib/python3.10/site-packages/tensorflow/python/autograph/impl/api.py", line 643, in wrapper
    return func(*args, **kwargs)

  File "/home/nonlin/miniconda3/envs/StepNet/lib/python3.10/site-packages/tensorflow/python/data/ops/from_generator_op.py", line 235, in generator_py_func
    raise TypeError(

TypeError: `generator` yielded an element of shape (32, 2, 15, 80, 3) where an element of shape (None, 2, 15, 80, 1) was expected.


         [[{{node PyFunc}}]]
         [[IteratorGetNext]] [Op:__inference_train_function_33834]

Trang chủ Giới thiệu Sinh nhật bé trai Sinh nhật bé gái Tổ chức sự kiện Biểu diễn giải trí Dịch vụ khác Trang trí tiệc cưới Tổ chức khai trương Tư vấn dịch vụ Thư viện ảnh Tin tức - sự kiện Liên hệ Chú hề sinh nhật Trang trí YEAR END PARTY công ty Trang trí tất niên cuối năm Trang trí tất niên xu hướng mới nhất Trang trí sinh nhật bé trai Hải Đăng Trang trí sinh nhật bé Khánh Vân Trang trí sinh nhật Bích Ngân Trang trí sinh nhật bé Thanh Trang Thuê ông già Noel phát quà Biểu diễn xiếc khỉ Xiếc quay đĩa Dịch vụ tổ chức sự kiện 5 sao Thông tin về chúng tôi Dịch vụ sinh nhật bé trai Dịch vụ sinh nhật bé gái Sự kiện trọn gói Các tiết mục giải trí Dịch vụ bổ trợ Tiệc cưới sang trọng Dịch vụ khai trương Tư vấn tổ chức sự kiện Hình ảnh sự kiện Cập nhật tin tức Liên hệ ngay Thuê chú hề chuyên nghiệp Tiệc tất niên cho công ty Trang trí tiệc cuối năm Tiệc tất niên độc đáo Sinh nhật bé Hải Đăng Sinh nhật đáng yêu bé Khánh Vân Sinh nhật sang trọng Bích Ngân Tiệc sinh nhật bé Thanh Trang Dịch vụ ông già Noel Xiếc thú vui nhộn Biểu diễn xiếc quay đĩa Dịch vụ tổ chức tiệc uy tín Khám phá dịch vụ của chúng tôi Tiệc sinh nhật cho bé trai Trang trí tiệc cho bé gái Gói sự kiện chuyên nghiệp Chương trình giải trí hấp dẫn Dịch vụ hỗ trợ sự kiện Trang trí tiệc cưới đẹp Khởi đầu thành công với khai trương Chuyên gia tư vấn sự kiện Xem ảnh các sự kiện đẹp Tin mới về sự kiện Kết nối với đội ngũ chuyên gia Chú hề vui nhộn cho tiệc sinh nhật Ý tưởng tiệc cuối năm Tất niên độc đáo Trang trí tiệc hiện đại Tổ chức sinh nhật cho Hải Đăng Sinh nhật độc quyền Khánh Vân Phong cách tiệc Bích Ngân Trang trí tiệc bé Thanh Trang Thuê dịch vụ ông già Noel chuyên nghiệp Xem xiếc khỉ đặc sắc Xiếc quay đĩa thú vị
Trang chủ Giới thiệu Sinh nhật bé trai Sinh nhật bé gái Tổ chức sự kiện Biểu diễn giải trí Dịch vụ khác Trang trí tiệc cưới Tổ chức khai trương Tư vấn dịch vụ Thư viện ảnh Tin tức - sự kiện Liên hệ Chú hề sinh nhật Trang trí YEAR END PARTY công ty Trang trí tất niên cuối năm Trang trí tất niên xu hướng mới nhất Trang trí sinh nhật bé trai Hải Đăng Trang trí sinh nhật bé Khánh Vân Trang trí sinh nhật Bích Ngân Trang trí sinh nhật bé Thanh Trang Thuê ông già Noel phát quà Biểu diễn xiếc khỉ Xiếc quay đĩa
Thiết kế website Thiết kế website Thiết kế website Cách kháng tài khoản quảng cáo Mua bán Fanpage Facebook Dịch vụ SEO Tổ chức sinh nhật