Keras 2 <> Keras 3 incompatibilities

[fchollet]

Keras 3 is a major new release. It features a number of cleanups and modernizations of Keras which leads to number of breaking changes compared to Keras 2.

The list below is exhaustive to the best of our knowledge.

A small number of items are likely to affect you (jit_compile default value change, TF SavedModel support changes, usage of tf.Variable as layer attributes). The majority are very niche. All APIs that were removed were dropped due to extremely low usage.

Behavior differences between old tf.keras and Keras 3 (with TF backend)

• APIs that were previously long-deprecated or experimental are gone. compat.v1 APIs are gone (deprecated in 2019). In the case of experimental APIs, usually those are APIs that have already moved to a permanent namespace long ago (e.g. the contents of tf.keras.layers.experimental.preprocessing is now at keras.layers, since 2021), so just update the import path to the up-to-date location.
• Keras 3 has jit_compile=True by default -- this might not work with all TF ops, so with some custom models/layers you might have set jit_compile=False if you see an XLA related error.
• Saving to TF SavedModel format via model.save() is no longer supported (note: you can use tf.save_model.save(model) instead)
• Loading a TF SavedModel file via keras.models.load_model() is no longer supported (note: you can use keras.layers.TFSMLayer(filepath, call_endpoint="serving_default") to reload any TF SavedModel as a Keras layer)
• Model() can no longer be passed deeply nested inputs/outputs (nested more than 1 level deep, e.g. lists of lists of tensors)
• In old tf.keras, TF autograph is enabled by default on the call() method of custom layers. In Keras 3, it is not. This means you may have to use cond ops if you're using control flow, or alternatively you can decorate your call() method with @tf.function.
• Using a TF op on a Keras tensor during functional model construction is disallowed: "A KerasTensor cannot be used as input to a TensorFlow function". Fix: use an equivalent op from keras.ops.
• Multioutput model's evaluate() method does not return individual output losses anymore -> use the metrics argument in compile to track them
• Layer names and variable names can no longer contain the / character.
• No RaggedTensor support. We may add it back later.
• When having multiple named outputs (for example named output_a and output_b, old tf.keras adds <output_a>_loss, <output_b>_loss and so on to metrics. Keras 3 doesn't add them to metrics and needs to be done them to the output metrics by explicitly providing them in metrics list of individual outputs.
• tf.Variable objects assigned to a Layer are not tracked as part of weights. Fix: use self.add_weight() method or use a keras.Variable instead.
• None entries are not allowed as part of nested (e.g. list/tuples) tensor arguments in Layer.call(), nor as part of call() return values.
• Functional models with list outputs do not accept dict losses/metrics anymore
• Symbolic add_loss() is removed (you can still use add_loss() inside the call() method of a layer/model).
• Locally-connected layers are removed (they had ~0 usage). Fix: copy the layer implementation into your own codebase.
• Kernelized layers are removed (they had ~0 usage). Fix: copy the layer implementation into your own codebase.
• Layer attributes metrics, dynamic are removed
• constants arg in RNN layers is removed (remnant of Theano, ~0 usage)
• time_major arg in RNN layers is removed (~0 usage)
• Removed reset_metrics argument from model.*_on_batch methods (~0 usage)
• RadialConstraint constraint object is removed (~0 usage)

Present in Keras 3 standalone but will work when accessing Keras 3 via the new tf.keras

• Various (undocumented) backend functions missing, e.g. backend.random_normal
• AlphaDropout layer is removed
• ThresholdedReLU layer is removed (subsumed by ReLU)
• RandomHeight / RandomWidth layers are removed (better use RandomZoom)

[AakashKumarNain]

Various (undocumented!) backend functions missing

I checked out the issue for converting keras.io examples to keras_core examples. One important functionality that is missing from the backend is CTC. Widely used but diff frameworks implement it in different ways. If we are expecting people to use keras_core for writing framework agnostic code, we need to implement it in the backend.

[PatReis]

Hello, I wanted to ask about the plans to add a KerasRaggedTensor to Keras Core. I would be very interested to have this feature, just to feed the Tensors of shape (batch, None, C, F, ...) to a keras model. I understand that keras backend ops can not support ragged operations across platforms but simply to pass a ragged Tensor to layers as input and output and extract splits and values would be super helpful.

[fchollet]

I would be very interested to have this feature, just to feed the Tensors of shape (batch, None, C, F, ...) to a keras model

We might support RaggedTensor in the future with the TF backend specifically (such a feature does not exist with other frameworks).

However if you're just looking to have a dynamic data dimension, then you can do it just by:

• Bucketing your samples into buckets of shape (batch, A, ...), (batch, B, ...), etc. and creating batches out of the buckets, so that each batch is rectangular
• Padding/truncating your samples to a shared shape
• Some combination of the two

In general it is possible to handle any workflow using rectangular tensors. RaggedTensors are a convenience but not a blocker.

[LukeWood]

I would be very interested to have this feature, just to feed the Tensors of shape (batch, None, C, F, ...) to a keras model

We might support RaggedTensor in the future with the TF backend specifically (such a feature does not exist with other frameworks).

However if you're just looking to have a dynamic data dimension, then you can do it just by:

• Bucketing your samples into buckets of shape (batch, A, ...), (batch, B, ...), etc. and creating batches out of the buckets, so that each batch is rectangular
• Padding/truncating your samples to a shared shape
• Some combination of the two

In general it is possible to handle any workflow using rectangular tensors. RaggedTensors are a convenience but not a blocker.

No idea how it is UX or performance wise, but torch recently added https://pytorch.org/torchrec/torchrec.sparse.html

“jagged tensor” instead of “ragged”

[innat]

A small number of items are likely to affect you ..., TF SavedModel support changes,

For backend agnostic, this may reasonable change (but quite big change). What about the .h5 format? Recently while trying to save keras model, I got the following, is it intended? What the diff of .weights.h5 or only .h5? I think I could do (.h5) earlier.

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
[<ipython-input-7-694b2dd940b3>](https://localhost:8080/#) in <cell line: 14>()
     12 model.compile(optimizer='adam', loss='mse')
     13 _ = model(np.random.rand(96, 96))
---> 14 model.save_weights('model.h5')

1 frames
[/usr/local/lib/python3.10/dist-packages/keras/src/models/model.py](https://localhost:8080/#) in save_weights(self, filepath, overwrite)
    371         """
    372         if not str(filepath).endswith(".weights.h5"):
--> 373             raise ValueError(
    374                 "The filename must end in `.weights.h5`. "
    375                 f"Received: filepath={filepath}"

ValueError: The filename must end in `.weights.h5`. Received: filepath=model.h5

In the built-in model, the .h5 for weight only looks still valid (though it's from tf.keras it runs w/o raising above error).

keras.applications.EfficientNetV2B0(
    include_top=True,
    weights="imagenet",
    input_tensor=None,
    input_shape=None,
    pooling=None,
    classes=1000,
    classifier_activation="softmax",
    include_preprocessing=True,
)
output
Downloading data from https://storage.googleapis.com/tensorflow/keras-applications/efficientnet_v2/efficientnetv2-b0.h5
29403144/29403144 ━━━━━━━━━━━━━━━━━━━━ 0s 0us/step
<Functional name=efficientnetv2-b0, built=True>

[burnpanck]

I was told over at tensorflow that Keras 3 will not support nested input dictionaries. While I don't understand why that support was removed, that's definitely an incompatibility that's worth mentioning here.