DQMH Consortium Toolkits Discussions

cancel
Showing results for 
Search instead for 
Did you mean: 

Submodules, VIPM and reusable code in offline environment

Hi all,

 

Although this is not a proper DQMH question I ask it here, hope it is ok (tell me if not..)

 

introduction: I am a single developer, working on a few projects in parallel. My local computer is a part of a closed network which does not have access to the internet. Recently I 'discovered' the SCC world and specifically Git, and I have now repositories in which I manage my projects.

As you understand I do not have the possibility to store my repos in GitHub, GitLab etc. In addition, I have some folders which I guess they fall into the definition of "Reusable Code". So after a lot of learning and reading (believe me...) I developed for myself the following method:

I created in our network drive a folder called "Repositories". In this folder I have bare repositories for each project and Reusable folder. This folder is like "My GitHub", in this sense that I can clone from them and push to them. When I want to use some of my reusable folders in a specific project, I create from the working directory a submodule that calls to the relevant bare repository.

 

And now to the questions...

1. What is the correct method to maintenance the Reusable folders? I understand that every submodule is like a reference to a specific commit in the folder. So when I make changes in the submodule does it correct to push it to the bare repository? In addition, Is it necessary to maintain the reusable fioder by itself? (If for example I will clone the bare repository into itself I will have all the files there. Do i need it?

 

2. I read a lot about VIPM, but I cannot understand how it helps me and if it an alternative to submodules or maybe I can combine between them (as Fabiola suggested in one example which I don't really understand..). specifically - if I create a package of my folder then the vi's are stored in vi.lib or user.lib, so all the changes will not be in the files that is in the repository. How do I SCC the files that way? and why do I need it?

 

3. Finally, does my method makes sense? Do I miss something?

 

Thank you,

 

Roey

0 Kudos
Message 1 of 7
(2,576 Views)

Hey Roey, it's good to hear that you found a way to make SCC work for you!

 

1. What is the correct method to maintenance the Reusable folders?

I fear there's no single correct method for that...

 

I understand that every submodule is like a reference to a specific commit in the folder. 

When you integrate another repository into your project repository by means of git submodule, git by default links it to a specific commit, just like you say.

 

Recent versions of git support tracking a specific branch within a git submodule. We at HSE did not really benefit from this (or don't understand it well enough), so we don't make use of this feature (see below).

 

So when I make changes in the submodule does it correct to push it to the bare repository? In addition, Is it necessary to maintain the reusable fioder by itself? (If for example I will clone the bare repository into itself I will have all the files there. Do i need it?

This depends on your way of working. At HSE, we usually only use git submodules when we want to modify the code inside the submodule from within the containing repository's application. You can find more about HSE's git submodule usage in our Dokuwiki.

 

For reusable code that can be developed and modified on its own (and which we don't want to modify from within a customer project), we would not integrate it as a submodule but rather create a package (*) and then use the released package in our projects.

 

(*) For us a HSE, a package can be 

- a source distribution which we copy into our project (usually for the more involved, HSE-specific reuse libraries)

- a VI package which we install into vi.lib (usually for the more generic reuse libraries)

 

In a nutshell, we want as many dependencies as possible in our project repositories (i.e. not installed somewhere else like vi.lib). More about this again in our Dokuwiki.

 

2. I read a lot about VIPM, but I cannot understand how it helps me and if it an alternative to submodules or maybe I can combine between them (as Fabiola suggested in one example which I don't really understand..). specifically - if I create a package of my folder then the vi's are stored in vi.lib or user.lib, so all the changes will not be in the files that is in the repository. How do I SCC the files that way? and why do I need it?

Using VIPM and VI packages gives you a number of advantages:

- building a package equals creating a proper, named release of your reuse library

- you can easily distribute your reuse library, others can easily install it

- source code installed via VIPM into vi.lib is usually not modified when working on a project - the fact that the reuse library is stored into vi.lib is a strong sign that it should not contain project-specific code (unless you consciously use it that way).

 

One way to do this would be to have the bare repo for your reuse lib on your network share and work on it by cloning the bare repo to your working directory (as you already do). Once you're happy with the state of your reuse library, you could then "create a release" (**) by building a VI package. 

 

In your projects, you would not include the reuse library via a git submodule, but you'd instead use it from vi.lib - i.e. using the VIs which you installed via your VI package and which now live in vi.lib.

 

If you need to add features to your reuse library or fix a bug, you would go back to the working directory of your reuse lib's repo, make your changes, build a new VI package, install it via VIPM into vi.lib, and THEN go back to your project and use those new (or updated) VIs. 

 

(**) There are many more ways to do this than only VIPM. You could mark specific commits in your reuse library as releases, eg by adding a git tag. That way, when you integrate the reuse lib into your project via git submodules, it is easier to find which exact commits you would like to actually track. 

 

Again, there is an abundance of possibilities to do all this. We have found a good way for us, and we even automated part of it (<- shameless advertising). 




DSH Pragmatic Software Development Workshops (Fab, Steve, Brian and me)
Release Automation Tools for LabVIEW (CI/CD integration with LabVIEW)
HSE Discord Server (Discuss our free and commercial tools and services)
DQMH® (Developer Experience that makes you smile )


Message 2 of 7
(2,547 Views)

Hi Joerg,

 

Thank you for your reply!

 

So if I understand you correctly, you prefer VI packages over submodules (can you explain to me what is the meaning of "a source distribution which we copy into our project"?).

 

I think (as you also wrote) that this option assumes you are satisfied with your reuse code. I am not sure this is the situation in my case because I see there are yet some changes that I need to do on them.. I thought there is a way to make submodule and package it and then manage the package versions with SCC (without using *.vipc files which I don't have). Is there such a way?

 

Roey 

0 Kudos
Message 3 of 7
(2,515 Views)
@roey_p wrote:

So if I understand you correctly, you prefer VI packages over submodules (can you explain to me what is the meaning of "a source distribution which we copy into our project"?).


Git submodules come with some level of complexity. Things can go wrong, and it's not always immediately clear to amateur users of git why things don't work or where the problem lies.

 

Also, it's good practice to maintain (i.e. modify, fix, develop...) your reuse libraries in a central place where it's easier to keep track of the changes, easier to avoid merge conflicts, easier to roll out new versions. This means that you should not modify your reuse libraries in your various customer projects - maybe even at the same time - because it makes it so much harder to keep track of the changes.

 

With all that being said, we use git submodules when we consciously decide to modify reuse code from within a customer project. In all other cases, we will create a new build specification for a source distribution in our reuse code's LabVIEW project file, which when executed will basically copy all source files to a separate location (like an export). You can optionally configure your build spec to apply a (random) password to your code when exporting it. That way, you cannot modify the code anymore from with your customer project.

 

As mentioned before, as a third option, we build VI packages from our more generic reuse code. This is an alternative to working with a source distribution. The advantage is that VI Package Manager offers lots of comfortable features for managing packages (it#s in the name!). The disadvantage is that code is installed centrally into vi.lib, so we cannot easily have different versions of packages in separate customer repositories at the same time.

 

I think (as you also wrote) that this option assumes you are satisfied with your reuse code. I am not sure this is the situation in my case because I see there are yet some changes that I need to do on them.. I thought there is a way to make submodule and package it and then manage the package versions with SCC (without using *.vipc files which I don't have). Is there such a way?

I think that would be the way we work with source distributions. You could use git submodules to include the "packaged" code into your customer repositories, but we don't do that. As described above, the complexities of git submodules are not worth the hassle in our opinion. We rather create release packages, host them centrally on our server, and copy new versions into customer project repositories manually. Again, we have built our own automated toolchain around our process, so this works very nicely for us.

 




DSH Pragmatic Software Development Workshops (Fab, Steve, Brian and me)
Release Automation Tools for LabVIEW (CI/CD integration with LabVIEW)
HSE Discord Server (Discuss our free and commercial tools and services)
DQMH® (Developer Experience that makes you smile )


Message 4 of 7
(2,496 Views)

OK, so now I have some more questions:

 

1. I actually never used the NI package Manager so after you suggested it I tried it for the first time, and I don't understand how it works...

    1) If I understand correctly the "Source Distribution" only create a folder that contains the files in which I want to package. Am I right?

    2) I created source distribution and then I created a package and installed it. what next? where do I suppose to see the package?

 

2. If I go back to VIPM: Suppose I have a project which uses the Reuse library as a VI package. Now I want to add things/fix bugs etc. According to your suggestion I should go to the central repo and fix it there and then create a new package and install it. But if I have another project which works well and I don't want to change anything in it - if now I make changes to the Reuse library I must know what is the version of the Reuse library before the additional changes.. how do you document this? with *.vipc file? (with Submodules I can push the changes to the repo and if I will not pull it in the 2nd project it will not affect this project.. Am I right?

 

Thank you,

Roey

 

0 Kudos
Message 5 of 7
(2,474 Views)

Roey, I fear that I've reached the limits of how much time I can devote to any single support case on the NI forums. 

 

Any of our DQMH Trusted Advisors would be happy to work with you on setting up your package/release strategy. Please don't hesitate to reach out to them, either through the DQMH website or directly.

 

Thanks for understanding,

J.




DSH Pragmatic Software Development Workshops (Fab, Steve, Brian and me)
Release Automation Tools for LabVIEW (CI/CD integration with LabVIEW)
HSE Discord Server (Discuss our free and commercial tools and services)
DQMH® (Developer Experience that makes you smile )


Message 6 of 7
(2,469 Views)

in answer to 2 - yes vipc files.

Sam Taggart
CLA, CPI, CTD, LabVIEW Champion
DQMH Trusted Advisor
Read about my thoughts on Software Development at sasworkshops.com/blog
GCentral
0 Kudos
Message 7 of 7
(2,453 Views)