GitHub First Class Integration with LabVIEW

[griffindore]

GitHub is amazing.

Figured that deserved to be on its own line just to let it sink in to the importance of this idea. Version control is no longer something only used by professionally trained software engineers. It's here for the masses; for the graphical design artists, animators, document writers. GitHub has a really cool web front-end to version control and brought sanity to DVCS (maybe bitbucket deserves some credit as well). It has championed a simplistic work-flow via pull-requests: GitHub Flow. It supports Issues for reporting bugs or other issues, and has an assortment of other collaboration features that make GitHub amazing.

GitHub (I don't mean git) integration with LabVIEW would demonstrate the power of Graphical Programming to the world.

Again, let that one sink in. There are 3 tenants of GitHub integration that LabVIEW must achieve:

1) Display the image of the block diagram and possibly front panel on GitHub when viewing a repository. Isn't it embarrassing that JKI State Machine Objects has to resort to putting images of VIs in the repo because GitHub can't render the VIs block diagram source?

2) NI must give away graphical diff and convince GitHub to run it. GitHub shows diffs on the web; Source Tree shows diffs in its tool. It's decision circa 1990 to not give Diff and Merge away with the basic version of LabVIEW. To think of diff/merge as advanced software engineering tools is a thought stuck in the past. LabVIEW needs its graphical diff shown within GitHub on a source file's history.

As a side note, vote for a version agnostic Diff/Merge idea here.

3) VI merge needs outstanding auto-merge capability that is built into pull-request merges. When creating a pull-request on GitHub, you'll see this statement (possibly), "Able to merge. These branches can be automatically merged." To work well in a DVCS multi-contributor, possibly open-source environment, the language needs superior auto-merge capability. Pretty much all other languages get it for free because they are text.

Keep in mind that NI will need a partnership with GitHub to accomplish this; however, this type of thing is not unprecedented on file type (maybe unprecedented with a proprietary langauge...). Just take a look at GitHub's ability to Render and Diff Images and GitHub's ability to Display Jupyter Notebooks (formerly IPython Notebooks).

[Enrico_Segre]

I use gitlab instead, but the same about integration could apply. Gitlab is already able to identify Labview as a language, see e.g https://gitlab.com/enricosegre/LVVideo4Linux/graphs/master/languages . Not to mention the dream of having an automated build server...

[griffindore]

On the JKI repo I linked, GitHub identified the langauge as 100% LabVIEW. I doubt GitHub, Gitlab, and Bitbucket will all get the attention. Since GitHub has been ahead of the curve in terms of Jupyter notebook rendering and image diffing and since I thought it was most widely used (couldn't find info to back up this opinion and I'm biased since GitHub is what I prefer), I thought it was the best target.

I've read items that alluded that Gitlab may be better for large sized repositories. If that's true, Gitlab is probably better suited for LV given the size of LV source files compared to txt source.

In terms of automated build server, that probably deserves another Idea post for LV+VIPM working with Travis CI. I think I saw that Gitlab has its own CI, but I don't use Gitlab much (only a few of my team's repos are there) so I couldn't tell you anything about it. The naysayers (not wanting to make it simple for open source projects) will say to run your own Jenkins server and set up the CI to run on a build node with a licensed version of LV. That's what my last LV project did that required CI although we didn't use VIPM.

[X.]

I wished I could intelligently vote for this idea, but I am afraid I am not sure how I would want this integration to work.

For one, binary formats for VIs should probably go, so that a simple character change on a FP of a 3MB VI doesn't result in my repository being incremented by 3 MB (I did not check that this is the case, but since VIs are compressed, I would assume so).

Visual comparison of the before and after state of a diagram (and the FP) would probably have to be done in LV (i.e. launch LV). I don't believe this would be practical to integrate within the multiple interface to github that are and will possibly be offered (I personally use SourceTree). However, I would see a possibility that LabVIEW provides a way to read the VI history and have this displayed in the "after" state. I am not against a snapshot of the diagram, but again, sometimes all that is changed is the FP, and in any case, both can sometimes be very involved. A visual comparator within github (or sourcetree or anything else) would be awkward.

[griffindore]

I purposely tried to leave it open-ended to allow NI to come up with their own solution instead of finding a specific aspect of my proposal that was flawed. However, your comment of doubt is enough for NI management to quickly cast this idea away as too much risk so here I go with a more detailed proposal.

First off, this idea is not tied to running LV Diff / Merge on a local machine's front-end to git (e.g. SourceTree). You can go the standard way of adding LV Diff and LV Merge to those tools. Remember that LV Diff and LV Merge cost extra money, and giving that away to be run in GitHub's or Bitbucket's cloud is different than giving it away to each individual user.

Second, I agree that NI needs to address 3 MB VIs as you called it; they typically aren't actually this large, but your point is valid nonetheless as LV source directories are monstrous compared to other langauges. We really need a truly "source only" version of VIs and a reduction of number of files (e.g. unpacked project library that has all the VIs in it as a single file on disk).

So on to the other items. LV already has a visual comparator: LV Diff. There's nothing that has to be awkward about displaying the result in GitHub. How could it be run? Well I don't know much about how GitHub works, but if each push to GitHub triggers an Elastic Cloud Compute (EC2) to run LV Diff and generate screenshots of the diagrams, these could easily be displayed in GitHub. Someone's gonna have to pay for the cloud computing resource, but it could also be a mechanism to charge per repo with this turned on.

Why do I say GitHub MUST display a diff? Well Pull Requests are at the heart of using a git remote. LV developers likely don't even know what they're missing, but I do a lot of coding in other languages and the pull request model is beautiful for multiple developer teams not to mention open source libraries that may be forked. If you can't visualize the diff on GitHub, then you can't have a meaningful collaboration over the code review. If you can't perform LV Merge, then multiple branch/fork doesn't really work effectively.

To start with, it would be nice just to be able to see an image of the LV block diagram (and possibly front panel) on GitHub. LV already supports snippets...

[X.]

FYI: http://forums.ni.com/t5/LabVIEW-Idea-Exchange/Future-VI-format/idi-p/3214605

[AristosQueue (NI)]

Just FYI: We have a working XML format for VIs in house. We've varied it in many different ways. NONE of the ways that we have varied it have made it possible to do a meaningful diff/merge with the text format directly. The problem is that LV diagrams express a graph of connections, so there are many many points where one section of the file has to refer to another section of the file (the <ConPane> section has to refer to the connections on the pane, but those connections are actually defined on the diagram or panel somewhere, so there's an index to refer). There are sufficient numbers of these (ones you might not think of like a data type table so we only write down one copy of every data type needed in the file and then have things refer to it) that integration of two files from the text is really impossible for any but the most trivial operations.

We continue to work on the problem, but our research thus far suggests that the information we attempt to record to define a VI is not amenable to the linear processing that characterizes most text languages and is what enables most of the diff/merge tools to work well.

[griffindore]

Just to clarify, this idea does not deal with, depend on, or recommend textual diff/merge of LV VIs. It's a recommendation of integrating the visual diff/merge tools that already exist into 3rd party ecosystems for DVCS.

[AndreasStark]

Any updates on this.... 6 years later and still not usable?

[joerg.hampel]

As Andreas says, it's been a while since this idea was first posted. At HSE, we use Git and GitLab extensively, and we have worked on creating open-source LabVIEW APIs both for Git and for GitLab.

In reply to @Enrico_Segre's "Not to mention the dream of having an automated build server...", we have a commercial product that helps with exactly that:

"Our tools help automate the testing, documenting, building, packaging and publishing of your projects. Built-in support for Git lets you trigger our tools from your repository, via GitLab CI/CD or Azure DevOps amongst others."

See https://rat.hampel-soft.com for more details.

[wiebe@CARYA]

As Andreas says, it's been a while since this idea was first posted. At HSE, we use Git and GitLab extensively, and we have worked on creating open-source LabVIEW APIs both for Git and for GitLab.

To me, that's a much better idea than letting NI do it. (So thanks!)

Git\GitHub\GitLab are just phases. When they pass, it take NI (another) decade or two to adapt to the next SCC hype. NI should provide the stubs, and (although designed for perforce IIRC), I think they're there.