[Discussion] 1.5.0 Roadmap

[szha]

Let's start a discussion here about the roadmap towards 1.5.0. We are looking for:

• New features that are useful to your research and development.
• Improvements and patches to existing features.

If you have any item that you'd like to propose to have in the roadmap, please do:

• Create (or locate existing) issue/pull request for the item, note the issue/pull request number.
• Comment in this issue: 1) the above issue number, 2) one sentence of what the item is about and why it's useful to you.
• Indicate whether you'd be willing to help out on the item.
• Share the ETA if you're driving the item and have an guesstimate on when it will be done.

cc @apache/mxnet-committers

[mxnet-label-bot]

Hey, this is the MXNet Label Bot.
Thank you for submitting the issue! I will try and suggest some labels so that the appropriate MXNet community members can help resolve it.
Here are my recommended labels: Feature

[szha]

The changes since 1.4.0 release that are already merged in the master branch will be included in the 1.5.0 release. The list can be found at: https://github.com/apache/incubator-mxnet/compare/v1.4.x...master?expand=1

[eric-haibin-lin]

Hi everyone, I've created v1.5.x branch here: https://github.com/apache/incubator-mxnet/tree/v1.5.x
Before we have an agreement on the timeline and features, I will synchronize this branch with the master branch periodically. Once we have decided the code freeze day, we will only cherry-pick required changes/features to the branch then.

[anirudh2290]

Thanks for starting this! I would like to include exception handling fixes: #14397 (@anirudh2290), #14433(@anirudh2290) , #14575 (@arcadiaphy). These three should be merged by end of next week hopefully. Conversion of FP32 models to mixed precision models (#14584) (Should be in by May first week tentatively). In addition, I have some changes to profiler to visualize gpu memory pooling and help make better decisions on the env variable choice. It is currently in a branch (https://github.com/anirudh2290/mxnet/tree/memory_profiler_poc2) and intend to open a PR soon (next week).

[pengzhao-intel]

MKLDNN Quantization PR

Name	PR#	status
sum	#14614	DONE
relu	#14604	DONE
refactor requantize	#14608	DONE
improve quantize	#14641	DONE
conv + activation	#14819	DONE
cache op	#14785, #14931	DONE
quantization flow to support 0 shape (RNN, concat)	#15031	DONE
New models (SSD COCO/RN18/MobileNet v2)	#14646, #14823	DONE

FP32 optimization

Name	PR#	status
data loader for CPU	#14824	DONE
transpose	#14545	DONE
RNN refactor with NNVM	#14476	DONE
reshape enhance	#14903	DONE
sum1d	#14914	DONE
softmax 1d	#14818	DONE
MKL Math (ERF, mean, etc)	#14893	DONE
MKLDNN RNN (vRNN,LSTM)	#14713	DONE
Build (Window/Linux)	#14740, #14743, dmlc/mshadow#374, #14829 #14877	DONE
Update MKLDNN to 0.19	#14783	DONE

Documentations

Name	PR#	status
Windows Build Instruction	#14952	DONE
MKLDNN OP	#14891	DONE

[zboldyga]

Some users pointed out useful features around matrix inversions, determinants, log determinants. I propose to add some small features to make these calculations easier: https://issues.apache.org/jira/projects/MXNET/issues/MXNET-1350?filter=allissues .

#14360

Comment in this issue: These are relevant calculations and some adjustments to the existing tools would help newcomers more easily leverage the existing work.

I'm interested and willing to implement this feature.

I'm quite busy at the moment but can likely finish this over a few days before mid May.

Thoughts?

[jmacglashan]

Easily the biggest feature Mxnet is lacking is the higher order gradient support. There appears to be some work to get this going, but it's been a bit stagnant. The lack of strong support for this feature prohibits the ability to implement a number of DL algorithms. Everything beyond this seems like quality of life features. I would offer to help on this front, but I won't have the time necessary to work it out. I list it here in hopes that others will answer the call.

Beyond that, I think having dynamic shape in symbols would be a nice feature.

On a smaller scale, I think it would be nice if Gluon had support for blocks that operate on keyword arguments. It's pretty easy to add support for that in a non-breaking way (and I've done it in my own projects), but ideally this feature would be supported in other code like the data loader, which currently is fairly structure around assuming tuples rather than dicts (which would pair with keyword args).

A nitpick that I have is that when it comes to serialization, Mxnet (python) seems to assume you always want to write to a file in that it requests a path to a file to serialize the data. This often isn't appropriate in production systems. It would be much nicer if Mxnet simply took a file-like object or just returned bytes so you can do what you want with it.

[KellenSunderland]

Features I'd like to see for 1.5 include:

Amp if ready

• MXNet AMP (automatic mixed precision) #14173
• ETA: Not clear

New TensorRT integration with subgraph API support and FP16

• Reformat of TensorRT to use subgraph API #14040
• Merged

NVTX ranges for easier GPU profiling.

• [MXNET-857] Add initial NVTX profiler implementation #12328
• Merged

+1 @pengzhao-intel on MKLDNN work. I'd love to make use of these optimizations. +1 to @anirudh2290's 3 very useful improvements.

[stereomatchingkiss]

Any plan to simplify compilation process on windows?
Any document to show us how to compile mxnet with the support of mkldnn on windows?

[pengzhao-intel]

Any plan to simplify compilation process on windows?
Any document to show us how to compile mxnet with the support of mkldnn on windows?

Yes, we have the plan for MKLDNN on windows and will fix it in 1.5. I will add into my table.
@yinghu5 @NeoZhangJianyu