[radcon]

I don't disagree, but isn't it a sign of much deeper issues if this "Shadow IT" land exists at your company in the first place? Why is it that experienced programmers can't efficiently solve a problem that a business user can handle in Access?

Are they under-staffed? Neck-deep in spaghetti code? Lacking business knowledge and the opportunity to acquire it? Bottle-necked by a lack of business analysts or testers?

I hear about this too often and nobody seems to think it's a major problem. IT departments can have a queue that's 2 years deep and people don't even bat an eye, they just think "Oh, that's how IT works!" And that's not even taking into account all the requests people aren't submitting because they've given up hope of getting any dev time.

[bigger_cheese]

The problem is most of the people writing code in Access (or monstrous Excel macros or in apps like MatLab etc) are not trained programmers. I think the article sums it up well by using "power user" to describe them - technical people with no formal programming training. Think Engineers (the non software kind), Accountants, Plant Technicians these kinds of people.

In my org (I'm an Engineer at a manufacturing plant) a long time ago someone senior made the decision our company was not in business of hiring programmers so any coding is done by contractors.

So yes people have implemented things (Engineering models for example) in MatLab, or Access, or as an Excel spreadsheet etc as "prototypes" ostensibly intended to be rewritten by a real programmer. At some point these models invariably find their way into production environment and suddenly you have some kind of mission critical thing written in Access (or VB6 or Matlab or Excel).

Then IT has minor panic a contractor is given the job of trying to translate the "prototype" into "real" code and has a battle on their hands.

Often programming contractor doesn't have technical understanding of what model is doing, regardless of choice of language the code engineers write tends to look like Fortran (i.e it's essentially all math formulas), even comments probably don't help when they say stuff like "/* Ergun's formula to calc Delta P */" that doesn't mean a lot to someone without a chem eng background. I've seen contractors produce bugs by doing things like misreading a spec and adding a variable called Fe203 (i.e Fe "two hundred and three) instead of Fe2O3 (Fe "Two Oh three" - a chemical formula). You can't really blame contractor he doesn't know anything about chemistry - he was just hired to rewrite this monster in java based on a spec.

So I don't think its a simple as turning problem over to an experienced programmer and expecting success.

[amostil]

I have seen all of these symptoms at numerous large companies. The backlog is years deep and the costs associated with engaging the professional programmers could be staggering. The truth is, not all solutions need anything more robust than what Office can provide.

[flukus]

Agreed, I think it's because developers like to over engineer everything and the managers like to over engineer the processes.

So many of these access excel solutions should be a days worth of work, simple perl cgi scripts with a minimalist UI deployed by rsync. Instead we have to use our super "productive" modern frameworks, split everything into a thousand files (god forbid you embed an sql query in the only place it's called), add unit tests, etc. There's certainly times for the later approach, but most businesses need much more of the former.

So because developers don't have a reasonable platform to pinch off random little projects others step in.

[perl4ever]

"So many of these access excel solutions should be a days worth of work"

I don't know why you think a lot of VBA would be replaced by a little perl. Of course, my perspective has a lot to do with the fact that it was essentially impossible to get a new perl module installed where I used to work.

But developers often don't appreciate the importance of presentation (and other) details in reports for managers. Your reference to "a minimalist UI" is telling.

Even though Access and Excel can be buggy, unstable, and annoying, it doesn't make much sense to use anything else if you are automating a report that was previously assembled by hand in Excel, and needs to match precisely.

And often IT types like to exercise power by gatekeeping - if you aren't doing "real" programming, you don't need a Turing complete solution, so Office ends up being the only option. I've been told that if I can select a list of columns from a dataset, and some filters, by pointing and clicking, that's all I, or my managers, need for reports.

Honestly, I think a lot of people find fulfillment in their work through being the person who can say "no" to people, particularly managers that are theoretically higher ranking. And also by expressing themselves through creative decisions when others fail to specify details. I think that using Office/Access/VBA may be correlated to rejecting the value system of most developers, rather than a technical judgment.

[flukus]

> But developers often don't appreciate the importance of presentation (and other) details in reports for managers. Your reference to "a minimalist UI" is telling.

We're talking access here, at best it's ugly and at worst you've got a gaudy background image(1) and a color scheme that would give the disability compliance officer a stroke. Throwing in bootstrap or a more minimal css framework is a huge step up in terms of presentation.

> Even though Access and Excel can be buggy, unstable, and annoying, it doesn't make much sense to use anything else if you are automating a report that was previously assembled by hand in Excel, and needs to match precisely.

I'm thinking of scenarios a bit more complex than that. Access apps generally have a few data input screens, multiple users, etc. Not complicated but not as simple as reports.

I'll admit that I do run away from anything to do with reports, but usually that's because they've installed some "easy to use, no developers required" reporting system that the non-developers can't use and makes life 10 times harder for the developers. If I can just write sql to shove data into an html table or excel template (where we can have the best of both worlds) I'm more than happy too.

1. I actually think of some of these old access programs when I look at windows new built in mail app, who the hell adds a background image?

[perl4ever]

"We're talking access here, at best it's ugly and at worst you've got a gaudy background image(1) and a color scheme that would give the disability compliance officer a stroke. Throwing in bootstrap or a more minimal css framework is a huge step up in terms of presentation."

I'm talking about creating Excel reports based on pulling stuff from (possibly a random assortment of) databases. Where you can use any feature of Excel. Not that the Access application is distributed to people who care what it looks like.

It seems like you can't even imagine a complex report that isn't an interactive application. So I think we're just talking different languages.

Honestly, I was just talking to someone in the organization I work in with the same lack of understanding. He was like "you have a point and click interface that lets you choose some columns from a table and some filters using simple boolean criteria, what else could you (or your manager) want?"

I want the ability to define all the business rules to produce the formatting and munge the data, I guess. And to structure the code in such a way that it's flexible enough to handle major changes. I need regular expressions. I need to run a diff algorithm on text. I need to use XML and REST to talk to SharePoint. I need to scrape information from a system that I only have access to through a web browser.

Basically, I'm using Access/Excel to do what I used to use Qlikview for, or just plain Perl, and it seems to be less of an "impedance mismatch" as people like to say. Also it doesn't cost as much as a car as Qlik licenses did.

[codeulike]

god forbid you embed an sql query in the only place it's called

-gasps-

[jotakami]

Because developing software is at least an order of magnitude easier and faster when the developer is the user?

[stblack]

This is so true.

[tabtab]

Re: Why is it that experienced programmers can't efficiently solve a problem that a business user can handle in Access?

Because existing web UI standards suck rotting ass to hell and back. The standard was not meant for CRUD and still isn't even if you add gajillion layers of (buggy) JavaScript to emulate a real GUI.

We! Need! A! Real! GUI! Standard!

[james_s_tayler]

Just master rails -g and you can be done in an afternoon.

[tabtab]

I agree that if you tune it for shop conventions and know its warts well, one can indeed be productive in it. But getting to that point is not necessarily short or easy.

[james_s_tayler]

I think that is a very fair and accurate assessment.

[cc81]

Sometimes it is budget or just needing someone to drive it through a heavy process.

It is also not always worth it if you cannot show a real business case. The thing might help you and it might be worth it if you hack something together yourself but if IT is suppose to build something that is available, has support, has disaster recovery, is patched etc. it might no longer be easy to find the business case for it.

[benhurmarcel]

Exactly, to get professional IT resources you need to demonstrate a business case, go through many decision boards and budget allocations, defend it for months, and be ready to be pointed out if it doesn't reach its promises.

So in the end people hack it in Excel or Access. No need to ask permission, no budget issues, no blame if it's not great.

[TeMPOraL]

Also: little or no waste happened if it doesn't pan out. Your Access prototype doesn't seem to make you and your team work faster? Copy out the little data you've put in there to Excel, SHIFT+DEL the Access file, and you're done. You've wasted few hours. That beats wasting several man-months of IT time only to not use the result in the end.

Access is agile in the truest form. It lets you prototype and refine in an extremely tight feedback loop, until what you have is a working solution for your specific problems.