[jac_no_k]

/me is one of the minions from team "Eye of Sauron". However I'm not stuck in this tower.

I'm torn between two worlds. One side applauds the grass roots based solutions and the JFDI attitude. The other side is concerned that these solutions becomes widely used, de facto production, and problems upstream ends up breaking the output. On error, hopefully fails loudly but subtle problems could creep in. If the reports are consumed by other automation, the problem spreads rapidly.

Ideally the IT department would take a step back, assess what the users (you the stakeholder) needs are, and provide the solutions. Better Agile.

Using the successes to drive more development and operations budget is the way to go.

[lbatx]

Until IT departments get infinite resources (that is, never) there will always be a need for these JFDI solutions. Some group will always go starved for attention and have to cobble together their own solutions. IT backlogs are always longer than the resources to do the work.

[edelans]

JFDI = Just Fucking do It

(in case other ppl wonder what it means... =) )

[nirav72]

Or what we call 'Shadow IT' in my department.

[kevin_thibedeau]

> Ideally the IT department would take a step back, assess what the users (you the stakeholder) needs are, and provide the solutions.

This approach was taken with Lotus Notes. The ivory tower decided who could create and control a database. All it does is throw up bureaucratic barriers people will work around. How much of your company is dependent on bespoke Excel databases? What better tools can you empower the employees with to mitigate that risk?

[jac_no_k]

@cosmie provides a good model for IT organization to quickly bring supportable solutions. The principles behind Agile is to reach across these barriers / towers. These Experts are the product owners / users, the developers, and also includes UX designers, operations team, security team, and capacity team. This is ideal but realistically challenging to pull off.

A given Firm should also be providing a way for Shadow IT project to brought into the light, formally adopted and supported.

[cosmie]

It's deeply flattering that you read to the end of my rambling comment, let alone referenced it! :)

I happened to be exposed to that model at one of my first professional jobs, and it was such an incredibly effective and seemingly obvious organizational structure that I presumed it was normal. Every job I've had since then has just deepened my appreciation for the brilliance and skill of whatever executive team pulled that off, as I've realized just how rare it is.

Fascinatingly enough, the organizational structure of that giant, traditional company bore a striking resemblance to how a software architect would approach designing a modern SOA on a cloud platform. And when the same principles of an SOA were applied to the design of the organization itself, the same communication structures, advantages, and trade offs emerged.

Which makes sense. Organizations and software are incredibly similar systems, and the parallels between the two allow for pretty seamless transfer of principles, best practices, pitfalls, mental frameworks, optimization opportunities, and reference architectures between organizational design and software design. Which is probably one of the hidden strengths of tech founders and the companies they create, since they'll implicitly infuse their experience at software design into the way they design all aspects of their company, even if they don't recognize they're doing it. Which is a competitive advantage over traditional business leaders that likely only have exposure to standard organizational design practices/teachings.

[cosmie]

I completely understand the terrors of hidden dependencies on shadow IT. The work I do tends to grow up organically in different areas of every org, so I've hopped between functional business units, centralized business operations, and IT. When working for business units, I'm careful when I introduce technically advanced process dependencies. I either limit it to the level of technical competency that is generally accessible to the team or else I adopt whatever toolset the IT or development side of the house uses. That way there's always the potential for transition and I don't screw my manager/team over if I leave (or get stuck supporting it in perpetuity while I'm there).

This was an example of that. We do client consulting work, but we're only about 2% of the total revenue of our Fortune 500 parent and our IT policies are not conducive to client-driven work requirements. When my manager looked for what we could use when supporting analytics-oriented aspects of client work, tools like Supermetrics[1] were explicitly denied and IT provided virtually no automation/analytics solutions short of a complete Tableau server, which is prohibitively expensive for most of our clients' needs (especially when allocating IT infrastructure & support costs in addition to the license). I'm not a fan of the fragility and monotony of repetitive manual data collection and reporting, nor do clients like to pay for the excessive hours that takes on a recurring basis compared to competing bids (because presumably our competitors can use SaaS tools for automating rote process).

I scrounged around and found a pocket of PowerBI users in our IT's SQL Server team. Connected with the director over that team and got access for myself. Created a proof of concept architecture that provided an agile platform to address the full spectrum of our potential needs from "throw in basic analytics for free and eat the cost" to "maintain a full on data lake getting a deluge of real time data streams". Client users are guests invited to our tenant, log in with their (usually) already-existing Office 365/AzureAD user, can have the $10/user/month PowerBI Pro license added to their account via their corporate IT, our corporate IT, or a credit card expense. All of their ETL and data connectors and credentials and PowerBI models are compartmentalized into their own app workspace and subject to corporate DLP policies and accessible/auditable by Office admins. It's technically unsanctioned/shadow IT, but I ran it by the director of that team in case I needed to tap his team for sporadic support needs in the future, and he was both interested in my proposed usage and perfectly fine giving it his blessing (which is great, because his team manages the permissions I need for provisioning users, adding licenses, and creating workspaces).

The alternative would have been to either lose business due to non-competitive bids, use a combination of unsanctioned SaaS tools that get expensed and subject to disruption when turnover happens and passwords don't get shared or credit card renewals don't occur (fairly common scenario I've inherited), or using development skills to create custom automation that would be unmaintainable when I left.

> Ideally the IT department would take a step back, assess what the users (you the stakeholder) needs are, and provide the solutions. Better Agile.

This works well in some cases, but falls flat on it's face just as often. It's often prohibitively expensive from a budgetary standpoint for a business team to pull in resources for a needs analysis and subsequent project. Business users are far cheaper, so the vast majority of drift between needs and systems capabilities/processes go unresolved because it's cheaper to absorb the inefficiency on the business labor for quite a while before it hits a critical point that justifies the expense of pulling in ops/dev staff. For most business teams, shadow IT is a pragmatic stopgap solution to hit their deliverables, similar to the type of tech debt you see in IT projects that gets kicked down the road until it's suddenly not something that can be kicked anymore. And the process of IT stumbling across a grass roots solution acting as a de facto production dependency is basically a litmus test for when the debt finally comes due and triggers a formal needs analysis and IT project.

As well, that type of needs analysis generally creates localized optimizations or poor user adoption. The current needs will be a reflection of the processes that evolved within the current system and constraints, and addressing the directly will usually just result in a solution that's "a faster horse". Or it goes to the other extreme, where the solution you provide is state of the art and it turns out they have no ability to transition from riding their current horse to piloting the new fighter jet you provided.

A far more effective approach is to creative a collaborative model. Grass roots based solutions exist for a multitude of reasons and won't ever go away. But the concerns of undocumented, unknown, and potentially unmaintained de facto production dependencies existing outside of the purview of IT is a very real concern. So create a migration path. Put in low friction processes, support structures, and policies that give power users on the business side the ability to create sanctioned grass roots solutions. It'll allow business teams to experiment with and build up competencies and informed opinions on newer capabilities and possibilities, allowing for more nuanced and effective needs analysis for major projects. And it also provides enough oversight and management for IT to step in when usage patterns indicate something has become de facto production and warrants a more formal transition to/evaluation by technical and operations stakeholders. And also provides valuable data to ops/IT on where people are needing to create grass root solutions in general, which can better inform their own future systems planning.