You've made similar comments throughout this thread. You don't seem to realize that "novel" does not have to mean universally novel. It could be something novel for the individual.
Recently I had one of these "boring" tasks at work. Set up simple CRUD API and hook it up to graphql for the front end to consume. Only the graphql server was implemented in the most strange way I'd ever seen, so figuring that out took longer than I thought it would, oh and devops had completely uprooted the way we connect to databases to increase our security.
The company is far from the last agile company I've ever worked at, but on large, complex teams, what I'm describing is incredibly common. You don't know what you don't know.
It only means you didn’t do your homework(no offense, everyone falls into this trap too and conclude the issue is that estimation itself is not possible ). You can’t give an estimate based on what the task would take at your previous job.
You need to call out what you don’t know(Now, don’t give me Rumsfeld wisdom.. Most software is not that complicated)
The homework is the job. You can only know how long it will take if you know what you’re going to do. And if you know what you’re going to do, why not just go do it? Just typing out the code doesn’t take that long.
It could be because you are working on a brand new area every single “Sprint “(or equivalent). That’s not prudent assignment of resources(the manager should know this).
I have worked in companies where ownership is somewhat static(same few folks work on a given area of the product) and things get done within 10% of estimates (a lot of innovation too).
I have also worked in terms where ownership is fluid and “resources” get assigned to tasks. Nothing is now “estimatable”. Your experience is probably in line with the latter.
Personally - a fair bit. :) At my last job I was hired to build something for internal use that wasn't available to buy anywhere commercially (a year after I started, some things started coming out that were lower quality and more expensive than what we had). Even after it was built, a lot of my work was mostly investigative to attempt to root cause things like nondeterminism, in systems that I didn't build and wasn't responsible for.
At my current job I'm on a team of generalists partially responsible for making sure work from every team is going to be usable by other teams - which can involve everything from just advising to being embedded in the other team for a few weeks/months (to simplify a bit).
Edit: both companies are in the self driving space.
Novel is often a matter of perspective. For example a company with no experience in GraphQL might want to integrate it, they don't know how long it will take because it's novel to them.
There's pretty much always something novel or unique in the way things are put together. I recently had a project that was at first glance a pretty trivial "enter data, verify it, package as XML and send to a backend". Until we realized that the combination of privacy requirements and an industry spec would have required us to do schema validation of the XML in the browser. And apparently the only Javascript library which
does that is machine translated from C, not very stable and doesn't support UTF-8.
O/T but was it XML Schema specifically, or just XML DTDs that you needed to validate against? For the latter case, there's [1] (my proj), plus the option to rewrite XSDs into DTDs (under natural assumptions eg no local element redefinitions) without having to wrestle with Xerces and/or libxml (which hasn't formally completed XSD support anyway) and emscripten. You might also have luck by transpiling Xerces/Java to JavaScript using Google's j2cl+closure-compiler (which works very well if you can swallow bazel and/or isolate the actual j2cl Java app from its bazel wrapper).
Recently I had one of these "boring" tasks at work. Set up simple CRUD API and hook it up to graphql for the front end to consume. Only the graphql server was implemented in the most strange way I'd ever seen, so figuring that out took longer than I thought it would, oh and devops had completely uprooted the way we connect to databases to increase our security.
The company is far from the last agile company I've ever worked at, but on large, complex teams, what I'm describing is incredibly common. You don't know what you don't know.