Skip to main content

Synapse

New

Fabric Environments and Python Custom Library Management

Vote (1) Share
Sérgio Torrinha's profile image

Sérgio Torrinha on 27 Dec 2023 09:21:14

Hi everyone!


For what I'm about to suggest, please refer to this post ( https://community.fabric.microsoft.com/t5/General-Discussion/Environments-what-s-planned-for-it/m-p/3601874#M3011 ) where I asked on what is planned right now, regarding environments.

 

Currently, and from my experience with MS Fabric so far, there are a couple issues and/or bugs with Environments that make its utilization hard:

 

  • When uploading new wheel files, with one's custom libraries, not always you can see the newly uploaded file. From my experience, this is because you have the environment being used in, at least, one of your notebooks. Therefore, you must assign another environment to your notebook, before you can do something with the env you were trying to edit.

 

  • When uploading new wheel files, you don't have an error message, whenever you try to upload a recent file and that upload fails - instead you get a message saying the upload was successful, but it shows your old wheel file

 

Sugestions here would be:

 

  1. Please improve UI messaging to user whenever there is something not possible to do, with a proper error message;
  2. Even better, would be to allow creation of environments that would not require the use of wheel files, because it's painful to recreate a wheel file whenever there is an adjustment that needs to be done on the custom library;
  3. Regarding point number two, I came to learn recently that there is a new feature, in my opinion poorly documented, that is ‘notebook resource’ which you can see it’s documentation nere: https://learn.microsoft.com/en-us/fabric/data-engineering/author-notebook-resource-with-vs-code . My suggestion is to improve the documentation and also the way this works currently: as far as I understood, the ‘notebook resource’ is nothing more than a set of python modules you can load into Fabric. It would be great if VS code would allow the automatic importation of the ‘notebook resource’ folder into Fabric (or a way to enable this), whenever there were modifications made to it;
  4. Related to VS code integration with MS fabric: suggesting here that it would be possible to enable a way to automatically download/publish notebooks and resources into MS Fabric. It can be cumbersome to keep pressing so many buttons to do all those functions and enabling an automated way of doing so, would allow to improve efficiency while developing this kind of solutions. Ans, yes, I do understand there might be some scenarios where you don’t what to automate these tasks, and that is why I am suggesting here an enabler and not a default behaviour, if you know what I mean.

 

Hope you find this message clear and helpful, otherwise, please let me know.

Thank you.