How to call MnistDataSet.read_data_sets

[pocky6]

Hi,

I have a sample code which used MNIST dataset, using

keras.datasets.mnist.load_data(); 

I updated drivers, cuda, cudnn, the packages to their latest version, changed OptimizerV2 to IOptimizer where needed. I get this error:

System.TypeInitializationException : 'The type initializer for 'Tensorflow.KerasApi' threw an exception.'
TypeLoadException : Method 'LSTMCell' in type 'Tensorflow.Keras.Layers.LayersApi' from assembly 'Tensorflow.Keras, Version=0.10.5.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' does not have an implementation.

I came to the docs (on readthedocs.io) which mentions the following code instead:

mnist = MnistDataSet.read_data_sets("mnist", one_hot: true);

But this method (read_data_sets) seems unknown from the current TF.Net source, which looks weird because it was a convenient way of getting it.
What is now the standard way of loading it ? Where is the actual updated version of the current version ?

Thank you in advance

[Wanglongzhi2001]

Hello, the following is a minimal example of training using the Mnist dataset , I hope it will help you.

TensorFlow.NET/test/TensorFlowNET.Keras.UnitTest/Layers/Rnn.Test.cs

Lines 61 to 79 in a95005f

	var input = keras.Input((784));
	var x = keras.layers.Reshape((28, 28)).Apply(input);
	x = keras.layers.LSTM(50, return_sequences: true).Apply(x);
	x = keras.layers.LSTM(100).Apply(x);
	var output = keras.layers.Dense(10, activation: "softmax").Apply(x);
	var model = keras.Model(input, output);
	model.summary();
	model.compile(keras.optimizers.Adam(), keras.losses.CategoricalCrossentropy(), new string[] { "accuracy" });
	var data_loader = new MnistModelLoader();
	var dataset = data_loader.LoadAsync(new ModelLoadSetting
	{
	TrainDir = "mnist",
	OneHot = true,
	ValidationSize = 55000,
	}).Result;
	model.fit(dataset.Train.Data, dataset.Train.Labels, batch_size: 16, epochs: 1);

BTW, since TensorFlow.NET has relatively few developers, its documentation is not very detailed, so I recommend you to read more test cases. ^_^

[pocky6]

Hi,
Thanks for this minimal example.
I saw these classes in the source of TF.Net project, but as it is slightly longer than the previous version ("keras.datasets.mnist.load_data();") I thought it was unlikely that it was the newest version, considering the documentation too (I mean, the documentation is useful and looks good, that's why I was a bit confused :) )
I'll use this one now, thank you very much :)

[Wanglongzhi2001]

BTW, I tested keras.datasets.mnist.load_data();, it doesn't have a problem. According to this error messageTypeLoadException : Method 'LSTMCell' in type 'Tensorflow.Keras.Layers.LayersApi' from assembly 'Tensorflow.Keras, Version=0.10.5.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' does not have an implementation., may I ask what is your version of TensorFlow.NET, and whether you use LSTMCell in your code?

[pocky6]

Hi again,
Before I mark your other answer as the answer, I have to say: I'm using TF.Net 0.110.0. I don't use LSTMCell anywhere in my code.
The exception I mention was just raised by calling keras.datasets.mnist.load_data();.
I'll test the code from your previous answer today, if it raises the same error maybe it's a mismatch between CUDA/cudnn versions ? I have no other hypothesis about why this exception is raised.

[pocky6]

@Wanglongzhi2001
Hi,
I tried to call these 5 lines from my project:

 var data_loader = new MnistModelLoader(); 
 var dataset = data_loader.LoadAsync(new ModelLoadSetting 
 { 
     TrainDir = "mnist", 
     OneHot = true, 
     ValidationSize = 55000, 
 }).Result;

The call never returns. I tried the same from "EarlyStoppingTest" in TF.Net source, and it returns correctly. I guess something prevents TF.Net code working from my project, and it's a bit weird. Anyway, if I find more, I'll tell you.

[pocky6]

@Wanglongzhi2001 Hi,
Some news from my question.
It seems I could use these 5 lines by respecting two points:

• not calling them from a ctor (even indirectly)
• not using .Result, but calling LoadAsync with await instead (this single change made my call successful this evening).
  Anyways, thanks for your answer, and for the work on that project. I had to dive a bit into its code and found it interesting.