[j-a-a-p]

VBA is the ultimate agile programming language. The company's IT aka Bureaucracy Department is stuck with Scrum, Squads and what not. And meanwhile in the other departments people are just getting things done with Excel/VBA.

Nothing has changed. In the last century this also happened and it was called islands of automation. In my bubble back then it was considered a good strategy, let departments first play around, and if they are on to something integrate it.

[sanitycheck]

I was talking about this with a friend the other week... I think what IT depts really need to do is let people go crazy with Excel/VBA, but write a script to monitor activity on xls files on the network over the long term.

If there's an xls which has been in regular use for more than 18 months, and it contains macros, then it can be assumed it performs some important role and should be properly documented and checked and could also be rewritten in a "real" language and officially supported. Set up a meeting with whoever made it, and whoever's touched it most. Approach it more like "we're improving your cool thing" than "we're taking away your toys".

[civilized]

No one at my company would ever let IT take over their Excel/VBA processes.

The moment IT touches your stuff, your job transforms from solving problems to writing emails and having meetings.

Any change, no matter how trivial, takes dozens of emails, dozens of meetings, and half a year to orchestrate.

If IT wants to help solve more business problems, it needs to fundamentally change its self-concept and purpose away from "prevent hypothetical bad things from happening at all costs" and move it towards "solve more business problems".

[analog31]

This is vastly underestimated. The difference between coding and managing coding.

[civilized]

Working with IT isn't even managing coding. Management implies power to hold someone accountable, while working with IT an extended exercise in nagging and supplicating an organization which is completely unresponsive and unaccountable in its outcomes and methods.

You might as well become an immigration lawyer and spend all day begging the government to explain why your latest M-10582-9DJVA-V isn't being processed in the normal time frame, even though it was stamped in triplicate and sent by Certified Mail with a full-color copy of every identification document you own.

The best way to deal with IT is to avoid depending on it ever in the slightest way. If you give it an inch, it will take a mile.

[gremlinunderway]

You're actually on to a really important thing that IT depts misunderstand about Excel/VBA monstrosities.

They exist because they work. You want UX or business analysis? You literally just got that done for you for free if you run into a Excel/VBA application. The hardest part of dev is figuring out requirements, so stop looking at these as toys and start realizing that shadow IT exists because of a gap in development. Full stop. You can argue all day long that you're working your assess off, and you do great products, but the existence of these apps is empirical proof that IT has missed the boat on developing something of importance.

Use that.

[lelanthran]

It might be sobering for "real" devs to find out that more "real apps written in real languages" go unused than apps in VBA.

[pjacotg]

Completely agree. I've always thought that Excel/VBA shows you where all the short comings are with your in house systems.

[arethuza]

"let people go crazy with Excel/VBA"

Many years ago a company I worked for used to send out a spreadsheet to its suppliers which they would complete with the products they offered and then when it was received back there was a button in the spreadsheet that would automatically upload the data to a central database.

When I first saw this I was curious how it worked and did a bit of investigation - turns out there was VBA behind the button that established the database connection and uploaded the data. What was amusing was that the user had hardcoded the database connection string including username and password. Of course this wouldn't work outside of the firewall - but I'd be careful about letting people get too crazy.

[bjornasm]

I have this great new product called "DELETE FROM products WHERE provider != 'mycompany'"

[xen2xen1]

Little Bobby tables!

[sancarn]

To be honest, not hard coding a connection string is quite tough too. It really isn't an easy problem to solve. And especially when every piece of software out there connects to data in different ways.

The reality of the situation is with proper IT support, there could be compiled Excel Addins which provide API connections to core systems such that proper authentication also takes place. But that requires a first step by IT. Either that or authentication via a web server to get a temporary connection string. Either way, it requires prior infrastructure.

[jimbokun]

Or, you know, just ask people what they're using Excel for, and seeing if there's a way you can help them improve on it.

[TeMPOraL]

> In my bubble back then it was considered a good strategy, let departments first play around, and if they are on to something integrate it.

Funny how with computerized process, IT departments are effectively central planners. The lowly workers get to only do what the IT secretariat allows. It is this way because national^Wcorporate security!

[jojobas]

It's not so much central planners deciding who can have what, but rather a natural monopoly. You don't want your water/electricity depend on a family-run shop that can just shut down, and neither you want your purchasing department to hinge on that guy from logistics who can just quit, leaving behind his magic incomprehensible spreadsheets.

[ethbr1]

We decided that the water/electricity utility doesn't get to control how you use the amount you consume.

Modern IT is more like if your water utility had final say over which faucet you installed and how you used it.

[bee_rider]

If you started using water in, like, industrial or agricultural quantities I guess the town would eventually get curious what is going on.

[ethbr1]

Well, you'd still be paying for it.

Authorizing use would be akin to the pre-Carterphone ATT model where only pre-approved uses would be allowed ('you can't attach your equipment to our network').

Thankfully, we eventually realized that was a dumb decision and moved to something closer to user freedom + network protects itself + zero trust.

Better to just guide behavior at the pricing level, and let people make their own decisions about use.

[bee_rider]

I think the analogy to water use just doesn’t work well.

If we had to make some sort of water use analogy, I’d go with something like; the corporate network is a somewhat protected environment that needs to be maintained to be useful. So it it is more like a reservoir than a faucet.

It would actually be OK for a couple people to go swimming and even pee in the reservoir. Some people could even boat in the reservoir, if they went out of their way to make sure that their boats are clean, safe, no pollution, etc. But lots of places just have a general “don’t go in the reservoir” rule. Not because a person would damage it, but because everybody doing it would.

It is hard as a residential user to use enough water to damage the reservoir, but hypothetically if you managed to, somebody would check in. Even if you are paying, the town doesn’t want to run dry. If there is a drought, residential users might be asked to use less water.

Price doesn’t work as a signal in corporate IT for individual workers, because it is expected that the company will “subsidize” the worker to the extent needed to do their job. If we want to make the analogy work—at least in some areas, landlords are required to provide water to their customers. In that case, you can use as much water as you want for free, but your landlord will get curious and might find some way to get you on the hook if you pass some reasonable threshold.

You can also do some things as a user like dump toxic waste down your toilet. This would be sort of like running a publicly visible unpatched XP system on the network. It would damage the system, and why do you have that in the first place?

Anyway, that was fun to write, but I don’t know that it is particularly useful. In order to make the analogy fit, we need to bring in as much complexity from the water management system as the IT system has.

[jojobas]

You can use as much water/power as the pipes/wires allow through, but you don't get to have petrol pipes, beer pipes or milkshake pipes laid to your property, neither you get to choose 160V DC or 430V 400Hz electricity, however useful all of these things are.

[TeMPOraL]

> but you don't get to have petrol pipes, beer pipes or milkshake pipes laid to your property, neither you get to choose 160V DC or 430V 400Hz electricity

Sure you can have all of these. They're just not offered as part of normal utilities. Nobody will care if you build yourself some, except maybe for petrol pipes due to fire/explosion risk.

[jojobas]

Good luck getting you council to approve buried pipes from a brewery even 100m down the road. Within the confines of your parcel - maybe, so enjoy your Visual Builder for (beer) Aficionados.

[7952]

Although those kind of questions around continuity happen across business. Sometimes the service you are offering depends on individual contact and flexibility rather than offering a commodity utility. And dependance on a small number of individuals is an acceptable and understood risk. The trouble with software is when managers don't understand that risk and offload it on another department when things go wrong.

[xen2xen1]

Oh dear God how many times I've had to support that.

[j-a-a-p]

I was wondering what would be replacing Excel/VBA after 3 decades as a citizen developer alternative. I could not think of something that even comes close. Any ideas?

[jimbokun]

There have been countless attempts, but the web seems strangely resistant to a low barrier to entry, high level GUI building tool.

Even though it's a visual environment, everything is strongly text based, from HTML to CSS to JSON payloads.

[specialist]

I'm very optimistic about Project Jupyter style notebooks. I believe, without any evidence, that they have much greater potential thoughout IT, devops, whatever.

Example: Imagine a CI/CD pipelines using notebooks.

I hate Jenkins/Hudson style build systems so much I could just spit. I just want to run a shell script.

(Alas, I haven't had the gumption to try this idea out yet. Soon.)

[jampekka]

Jupyter style notebooks are already becoming the next VBA in some fields. And this is not a good thing.

[j-a-a-p]

I guess that some developers requirements are orthogonal to those of citizen developers. For example version control could be a must have for us, but for others a layer of complexity that is just waiting to stand in the way of getting things done.

[jampekka]

Yes. Jupyter et al are quick to get started with and you can get quite far until things start to get unmanageable.

But this may not be good for anybody in the long run. For example it tends to lead many students to not understand basic concepts, like variables. Which is understandable because variables don't behave like variables in notebooks (e.g. the same variable in the same notebook may refer to different values in different cells depending on how they are run). For many students this can cause almost insurmountably wrong mental models (which they will of course carry to "production" later on).

But as I argued in another thread here, it doesn't have to be this way. E.g. Pluto does notebooks in a more rigorous manner.

Almost all "software engineering" languages and tools makes getting started and actually getting something done quickly needlessly difficult. Probably uncontroversial that git UI is a total mess, and things are getting even worse with more build tools, dogmatic static typing and general pointless ceremony.

[specialist]

Why? The usual suspects? Lack of version control? Hard to deploy (reproducibly)?

[jampekka]

Yes and yes. But the larger and more fundamental problems are the mixing of the program logic and the state and inability to make the code composable or modular. Problems in version control and deployment/reproducibility almost necessarily follow from these.

These are probably not impossible to solve for notebook-style, but there are not many efforts to solve them or they are not even acknowledged as problems.

Edit: There is Pluto for Julia that attempts to solve the state-problem. I have not used it in practice though; I've given up on Julia, in large part because Julia community tends to be even actively hostile towards "stateless" development.

[specialist]

Thanks. Agreed.

By "stateless", I'm assuming you mean functional programming paradigms of immutable, idpotent, and no side effects.

FWIW, for build pipelines, my quarter-baked notion is to use ZFS snapshots (or equiv).

I'll check out Pluto for Julia.

As you know, state is a challenge for "serverless" too.

I've been reacquainting w/ RDBMS tools. There are a few new strategies (implementions) for change tracking. Back in the day, we just banged the rocks together (ook, ook), so I'm very eager to learn the new hotness.

[pjacotg]

I work in banking and I've noticed an uptick in Python where a decade ago it would have been VBA. Still plenty of VBA around but Python is in the mix as well.

[culebron21]

You nailed it, haha! I had exactly the same experience: I was working in 2007 in analytics department, and we had no IT bureacracy, meanwhile the IT department, we had to deal with, would say it would need months of approval for anything substantial (like a simple dashboard or a report), and a year to implement.