Skip to content

Lazy loading #326

Closed
Closed
Lazy loading#326
Assignees
Nucs
Labels
further discussNeeds discussion before implementation can proceed

Description

@aidevnn
aidevnn
opened on Aug 1, 2019

Hi
Your lib NumSharp was inspired me and i tried to write a other approach of NumPy more easy to use.
The big idea is the Lazy Loading for the front end and this is my repo.
https://github.com/aidevnn/DesertLand

My lib isnt optimized for speed at this time, but i can improve it with ease by rewritting NDarray backend class and methods WITHOUT changing nothing from the NDview struct frontend which use the lazy loading.

Actually your Backend is very very powerfull and maybe the frontend can be also improved with Lazy Loading.

Best regards.

Activity

  1. Oceania2018
    assigned
    Nucs
    on Aug 1, 2019
  2. Oceania2018
    added
    further discussNeeds discussion before implementation can proceed
    on Aug 1, 2019

  3. Oceania2018 commented on Aug 1, 2019

    @Oceania2018
    Oceania2018
    on Aug 1, 2019
    Member

    @aidevnn Join us, build Deep Learning library based on current work.

  4. aidevnn commented on Aug 1, 2019

    @aidevnn
    aidevnn
    on Aug 1, 2019
    Author

    I started to fork your repo, it takes me some month to understand in deep your great work.
    Best regards.

  5. aidevnn
    closed this as completedon Aug 1, 2019

  6. Nucs commented on Aug 1, 2019

    @Nucs
    Nucs
    on Aug 1, 2019
    Member

    Hey @aidevnn, Thank you.
    A. Are you aware we are rewriting our backend in a separate branch?
    B. Can you elaborate more about what do you mean by a front-end lazy loading?
    If I understand correctly, lazy-loading is something that numpy does not implement therefore if you wish to contribute a lazy-loading front-end when NumSharp is the backend - it'll have to be in a seperate project.

  7. Nucs
    reopened this on Aug 1, 2019

  8. aidevnn commented on Aug 1, 2019

    @aidevnn
    aidevnn
    on Aug 1, 2019
    Author

    Hello @Nucs

    A. Ok a devel branch is more collaborative than a fork

    B. Lazy Loading can separate the SharpNum library onto frontend with simple to write specification which will be used by the rest of SciSharp ML/AI framework and the pure SharpNum backend for fast and speed computation. The SciSharp ML/AI team can write their own unit test to integrate SharpNum progression.
    Lazy Loading is a design pattern wich allow to build complex expressions and he acts sometime like -proxy-. It is also a possible abstraction for integrating multiple different backend but it isnt a good idea at this moment for the framework SciSharp.

    For example, the expression
    NDproxy d = a+2*b+np.log(c);
    NDarray e = d;
    will only be evaluated during the implicit type conversion.
    I am a self taugh and i discovered the lazy loading just now to reduce computation. I have some intuition about some other usage of the Lazy Loading, but i cannot explain it.

  9. Oceania2018 commented on Aug 1, 2019

    @Oceania2018
    Oceania2018
    on Aug 1, 2019
    Member

    @aidevnn Is NDProxy like a kind of Expression/ Func/ Delegate ? It can be executed after all operations have been decided ?

  10. aidevnn commented on Aug 1, 2019

    @aidevnn
    aidevnn
    on Aug 1, 2019
    Author

    @Oceania2018 Yes it is a struct that can be defined like this during internal computation of the backend

    NDproxy a = new NDproxy(()=>ndArray0)

    Its contains only one field / property :

    Func<NDarray> fnc

    which can be called explicitly

    NDarray b = a.fnc()

    during internal computation of the backend
    or implicitly during type conversion by the frontend.

    In my repo i used delegate + struct.
    https://github.com/aidevnn/DesertLand/blob/master/NDarray/NDview.cs#L4

  11. Nucs commented on Aug 1, 2019

    @Nucs
    Nucs
    on Aug 1, 2019
    Member

    @aidevnn
    A. You can still use that branch from a fork
    B. Thanks for laying it out for me, I am too a self taught.
    Honestly - there is no fast way to stack lazy loading ops. Any basic implementation in C++ will surpass C#'s performance by far.
    Your best luck is to put efforts in Tensorflow.NET since tensorflow holds a somewhat similar API to numpy and is capable of computation on a GPU and it provides a pretty impressive performance (both CPU and GPU).

    if you do insist on writing it on your own then you better get started with IL generation that will link the math ops together.
    Mainly because every math-op method you delegate to the next math-op method is adding more overhead than you would think.

    NumSharp itself is not the fastest it can be. Any library that utilizes the performance of a low-level languages will be faster than our unmanaged algorithms.

  12. aidevnn commented on Aug 1, 2019

    @aidevnn
    aidevnn
    on Aug 1, 2019
    Author

    @Nucs you are right! Lazy Loading adds some stacks operations because its a kind of managing expression pattern.
    Managing code have advantage for simplifying coding and maintenance, but with loss of speed and performance. Its a tradeoff.

  13. aidevnn commented on Aug 2, 2019

    @aidevnn
    aidevnn
    on Aug 2, 2019
    Author

    @Nucs The idea behind the lazy loading is to introduce step by step some symbolic neural network algorithms with the numerical algorithms.
    For beginning, a precompilation (or may be caching) of all requirement like shape / strides / slices can be done without recreating them each time and its my first objectif. I will let you being informed on my progress

  14. aidevnn commented on Aug 3, 2019

    @aidevnn
    aidevnn
    on Aug 3, 2019
    Author

    I forget to comment an important thing about some counters.

    var a = ND.Uniform<double>(1, 10, 4, 4); // Counters. DataAccess:16 / MethCall:16
var b = 3 * ND.Sq(a - 1) - 4; // Counters. DataAccess:16 / MethCall:80
var c = 3 * ND.Sq(a - 1) - 4 * ND.Sqrt(a + 5); // Counters. DataAccess:32 / MethCall:144

    In the above expression, my actual code and symbolic approch reduce rawdata access only to one time for expression b. Also in a more complexe expression, the lazy loading reduce the data access to the minimum effort for expression c. But i am trying to solve some performances problems.

    We can say that a lazy loading pattern can improve the performance in theory but it introduces some other execution stack problems. I will continue to search how to take advantage of it.

  15. aidevnn commented on Sep 25, 2019

    @aidevnn
    aidevnn
    on Sep 25, 2019
    Author

    I discovered in this article
    https://techdecoded.intel.io/resources/parallelism-in-python-directing-vectorization-with-numexpr/#gs.63gvwl

    Numexpr : it is very usefull and it already use lazy-loading and a lot of symbolics optimisations to enhance performance.

  16. Nucs commented on Sep 7, 2026

    @Nucs
    Nucs
    on Sep 7, 2026
    Member

    @aidevnn, partial lazy loading shipped in 0.70.0 (#628): np.load(mmap_mode:) returns a zero-copy memory-mapped array, and NpzFile is lazy + cached:

    // read-only mmap - the file is mapped, not read into memory up front
var a = np.load("big.npy", mmap_mode: "r");
Console.WriteLine(a.flags.writeable);   // False

// "r+" maps read-write (writes flush to disk); "c" is copy-on-write

    Closing (a full lazy-evaluation graph is a separate, larger effort).

  17. Nucs
    closed this as completedon Sep 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

further discussNeeds discussion before implementation can proceed

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions