[atomicnature]

I apologize for not addressing your concerns point by point, but here are some general thoughts:

The kind of reasoning you are engaging in now was done by Descartes long ago, during Newton's time. It involves building up knowledge from the bottom up, starting with first principles. Descartes had some genuinely interesting ideas about how perfect concepts could be constructed. You can read his book on methods; they are remarkable, but they do not tell the whole story. Descartes himself made many scientific errors, despite his "perfect" and "rational" system for arriving at "true and sound conclusions." Popper's falsifiability is not the end-all-be-all of scientific thought either. Both Descartes and Popper do not represent the final word on what science is or should be.

Unlike Descartes, we observe real scientists supplementing rational ideas with empiricism—how actual people learn and think in their daily lives. How they go about life matters. Especially with Large Language Models, we see how essential a large knowledge corpus is for generating new variations. Marvin Minsky referred to it as "common sense," etc. It's important to remember that the concepts of "Expert systems" and "heuristics" did not work on their own back in the day (though they are not without utility in enhancing new methods).

Progress typically requires a societal-level effort in any field. It involves communication, challenges, numerous guesses, trial and error, and so forth. Therefore, the development of new ideas and the exploration of new frontiers are deeply interdependent at the civilization level.

In summary, I believe that "individualism" is incompatible with reality; the true nature of reality is "interdependence." Dependency is a fact of life. However, due to certain surface-level cultural ideas and the significance people attach to their self-importance, the unrealistic concept of "individualism" often prevails at the linguistic level over the more realistic idea of "interdependence."

[chromoblob]

> Especially with Large Language Models, we see how essential a large knowledge corpus is for generating new variations.

I don't see it, sufficiency doesn't entail necessity.

> It's important to remember that the concepts of "Expert systems" and "heuristics" did not work on their own back in the day

Expert systems are about automation of decision. I talk not about automation, but just a minimal, basic dependency on culture as opposed to complete dependency and no hope of novel thinking outside of it.

> Progress typically requires a societal-level effort in any field.

I want to know how this requirement would be quantified. Which society do you need for a given amount of progress?

> "individualism" is incompatible with reality

Why can't one, in principle, recreate some design on their own that they lack, of less complexity than rocket science or AGI?

[atomicnature]

> complete dependency and no hope of novel thinking outside of it.

Where have I said "there is no hope of novel thinking out of it"? (Re: Most thought is borrowed) I am simply saying not a single thought is entirely yours. There's always something external mixed in, stuff from society. Thought is always dependent. That means, you can be the co-author of an idea, but not the sole author. Having sole author is merely a simplification device, not a realistic description of the situation.

By the way, I still have not made the ultimate challenge here (since this is HN, filled with no-nonsense, individualistic hackers): what is referred to as "I", is also an idea delivered by society. Therefore, this whole conversation is thought talking to thought, and not chromoblob vs atomicnature.

[chromoblob]

Okay, since to develop the thinking skill, one needs to exercise a lot in a man-made environment, I agree that thinking is dependent on it and the basic vocabulary.

However, the dependency ends here. If you just want, you can analyze, deconstruct, modify or synthesize any memes at any level beyond the level of your primitives (given enough time).

About the definition of boundary of mind, I agree that it should be arbitrary and minds are compositional in space.

[atomicnature]

> However, the dependency ends here. If you just want, you can analyze, deconstruct, modify or synthesize any memes at any level beyond the level of your primitives (given enough time).

Thought Experiment: Say an isolated child was given the primitives of your choice, and nothing more. What's the likelihood that it'll invent calculus on its own during its lifetime? Or any other significant human discoveries/inventions. Will it figure out human flight on its own?

Leave that. Say there are no teachers/industry, except just all the textbooks. Still what is the likelihood of it teaching itself calculus or how to build a flying machine based on mere reading material?

[chromoblob]

> Thought Experiment: ...

Those may be too complex to invent from scratch in a few decades.

Long intellectual collaborations are a common good. You don't need to consider them estranged from their users, though. By virtue of emotional audit, knowledge I accept is assimilated by me, becomes mine. It was made by my allies, essentially by past instances of myself - nothing external here. (More like allies if it is precursor that I had to non-trivially modify for my needs, and more like self if it is a product that I accepted verbatim.) This situation doesn't contradict my sense of individualism. (In my view, personal identity is defined by preferences. So if we have same preferences, we are one. And it seems obvious that people with similar preferences will create functionally similar designs.)

So even if a child would in principle want calculus but wouldn't create it in a lifetime, they would take the steps they can in the direction that they're interested in. The distance would be dependent on capability, but more than "just a few ideas", as you say, wouldn't be improbable. Then the child may try and appoint those that the child seems worthy of receiving child's work, or child may publish it for everyone. That someone may deceive the child about self is an unsolvable flaw here. This may change in future when people solve old age mortality.

> Say there are no teachers/industry, except just all the textbooks.

To use books, one needs the ability to read. Without somebody to teach one to read, there is a small chance of success if a book that teaches written language and supposes no knowledge of written language (in usual meaning) is created. It would rely on a way to read the book and learn that would be figured out by the person, and this would work only if the person were very curious. Extent to which written language would be learned would depend on how well the interaction with the book works and how smart the person is; probability of the whole thing working seems very low, but depends on those too. Speechless video game tutorial theory would be mainly applicable here, only the medium here is also much more limited.

When you can read, I believe that text is sufficient to transfer knowledge. Quality subject books together with some books about effective learning may be sufficient, however, even if one finds a big library with all the knowledge early in life, they may simply not know what they want in terms of the index of the library, a guide book in front of everything may solve that. I don't know if a lifetime is enough to actually build a flying machine given only raw materials.

[atomicnature]

> Those may be too complex to invent from scratch in a few decades. Long intellectual collaborations are a common good.

Yes, that is the source for my assertion that culture/collective thought/pre-computed results is of primary importance ("essential"). You need way more than mere primitives to get upto any useful level of knowledge in a given lifetime.

> By virtue of emotional audit, knowledge I accept is assimilated by me, becomes mine. It was made by my allies, essentially by past instances of myself - nothing external here.

In other words, you are "sampling" something from the common culture based on an "emotional audit", and then tweaking the sample to suit your "preferences"? In other words:

P = Bundle of preferences (sampling biases that an emotional audit will check for)

I_1 = State of Identity

T = Transformation function that converts a sample of common good to suit the bundle of preferences

I_2 = State of Identity after integrating the result of transformation

I_1 + T(P(CG)) => I_2 (your formula)

i.e previous_identity_state + transform(select(common_good)) = new_identity_state

You argue that if, transform(select1(common_good)) = select2(common_good), then select1 = select2 i.e preference1 = preference2 [1]

The problem with this model is that it is dualistic; you envision immutable (?) preferences and mutable identity states (and obviously mutable common good which is a super set of identity states and 'other stuff'). The common good is getting manipulated in a distributed and concurrent way all the time, so even if the preferences remain immutable, the common good keeps changing, and therefore [1] wouldn't work out (the actual equation will have common_good1 and common_good2)... I think all dualistic ideas of defining the mind tend to get into this sort of trap.

That is, the following will not remain true ("create functionally similar designs") due to changes/updates in the common good (especially across time):

> So if we have same preferences, we are one. And it seems obvious that people with similar preferences will create functionally similar designs.)

I think a better model is to consider all localized preferences as part of the larger common good. There is just one common good (due to "dependency" of thoughts), across time and space, out of which everything emerges ("collective thought"). Such a model wouldn't admit any sort of immutable identity within the realm of thought.