So, you build a web app and it gets popular. It needs one load balancer, 5 app servers, at least two database nodes for replication, a redis cluster for caching and queuing, an elasticsearch cluster for full text search, and a cluster of job worker servers to do async stuff like processing images, etc.
In the ancient past, like when I'm from, you'd write up a few different bash scripts to help you provision each server type. But setting this all up, you'd still have to run around and create 20 servers and provision them into one of 5 different types, etc.
Then there's chef/puppet, which takes your bash script and makes it a little more maintainable. But there are still issues: huge divide between dev/prod environments, and adding 5 new nodes ASAP is still tedious.
Now you have cloud and container orchestration. Containers are like the git repos of the server world. You build a container to run each of your apps (nginx, redis, etc), configure each once (like coding and committing), and then they work identically on dev and prod after you launch them (you can clone/pull onto hardware). And what's more, since a container image is pre-built, it launches on metal in a matter of seconds, not minutes. All the apt-get install crap was done at image build time, not container launch time.
Things are a lot easier now, but you still have a problem. You're scaling to 30, maybe 50 different servers running 6 or 7 different services. More and more you want to treat your hardware as a generic compute cloud, but you can't escape that, even with docker, your servers have identities and personalities. You still need to sit and think about which of your 50 servers to launch a container on, and make sure it's under the correct load balancer, etc.
That's where Kubernetes steps in; it's a level of abstraction higher than docker, and works at the cluster level. You define services around your docker containers, and let Kubernetes initialize the hardware, and abstract it away into a giant compute cloud, and then all you have to do is tell kubernetes to scale a certain service up and down, and it automatically figures out which servers to take the action on and modifies your load balancer for that service accordingly.
At the scale of "a few servers", Kubernetes doesn't help much. At the scale of dozens or hundreds, it definitely does. "Orchestration" isn't just a buzzword, it's the correct term here; all those containers and services and pieces of hardware DO need to be wrangled. In the past it was a full time sysadmin job, now it's just a Kubernetes or Fleet config file.
Disclosure: I'm currently writing a book on Docker. Disclaimer: I have not had my coffee yet.
Edit: Since someone asked, I'm writing a book called "Complete Docker" which will be published by Apress. I don't know the exact pub date that Apress will launch it on, but I expect it'll be available in October.
Given the quality submission, I feel like it would be fair to include a link to the book your writing (or a title so we can search for it when its done). Disclosure: I want to buy it.
Wau thanks, that is an excellent explanation. I appreciate that it start with things I know and understand ("so you build a web app"/"bash scripts"/"puppet") and builds on that explaining what problems each consecutive steps /layer of abstraction/ solves.
Now I wonder.. how many projects actually needs these kind of solution when even StackOverflow can do without it (they are in the range of few servers)? I would imagine it would be only few top popular web apps/services, but by popularity of these posts it looks like it is probably a lot more...
How many people actually need it? Nobody! We managed to run servers for years without containers or orchestration tools. Docker is a new technology.
Does it make things better, though? Yes. Yes it does.
Kubernetes was designed at Google where they reaaaaally feel their scale problems day to day.
I see Docker/Kube/CoreOS/ etc as the natural evolution of where we were already going. Bash -> puppet -> vagrant -> docker -> kubernetes. Less abstract to more abstract.
So it's actually "only" an incremental evolution in terms of managing the server ecosystem. But it's a revolutionary improvement in how we think about server ecosystems, which is why many people struggle with Docker et al at first; it's a brand new mental model.
I'd like to add that this isn't necessarily even about huge scale. It's about scaling deployment patterns too, which is different than scaling services.
Sometimes you need to have the ability to run more services when your "web-scale", but even if you're not at that point, scaling how you deploy new versions, and new services is still important.
Tools like docker, and Kubernetes really help with the application delivery aspect, and really enable you to rapidly iterate on your project.
You might have a site with 100-1000 users, not huge by any means, and 1 server could probably handle all of your needs. But once you start adding other components, perhaps redis, or a runtime like nodejs, those can all be managed, but if you need to rapidly iterate, something like Kubernetes/docker can make updating or deploying these easier in the long run.
Very few actually need them, but many companies have scale-envy and instead of getting bored with their tech stacks, stay inspired/passionate by playing with the shiny new tech running their low/medium traffic website.
In addition to bkanber's comment, this technology is helpful sooner if you're using a microservices architecture, where you have very many small servers instead of a few beefy ones. Instead of "Time to spin up a new box" it might be "Time to spin up 3 box A's, 2 box B's, and a box C, D, and E...".
I think it's clear that the vast majority of profitable businesses, online or off, don't need the scale these services provide. But providers to these business can benefit from the scale to provide better, cheaper services. For example, the resurgence of the Static Web: ie site generators (ie Jekyll) and vendors offering hosting for these services (ie bitballoon, netlify, aerobatic, neocities, etc...).
Really, why should cost anything to have basic, low-traffic, site on the internet today? That's the quetion I ask myself.
Containerization on it's own can have benefits even in smaller deployments - separating your different server concerns to the bare minimum can be very useful.
There's also other aspects, like being able to spin up the newest PostgreSQL within minutes, and when you're done you rm the container and image and have no trace and/or side effects.
True, although I'd say most users are logged in, so that does make caching much harder.
stackoverflow is just a nice example that you don't need dozens of servers, even when you reach their scale.
I do believe Brook's The Mythical Man-Month does also applies to computers. In that communication overhead soon outweighs the benefits of adding more computer nodes.
> All the apt-get install crap was done at image build time, not container launch time.
Something I never understood with containers: where do they store persistent data, e.g. MySQL's /var/lib/mysql - and how does upgrading work, i.e. when the apt-get postinstall script runs transformation on the persistent data, how is the transformation applied to the "clones"?
"Volumes" are the Docker construct used to store persisted data. You use a volume when you want to decouple the lifecycle of your data with the lifecycle of the app. You can either map the volume to a directory on the host (ie, map /var/lib/mysql to ~/data/mysql), or you can allow Docker to manage it (where it'll live in /var/lib/docker/volumes/blah/blah).
You don't upgrade a running container. Imagine that containers are immutable; to launch a new, upgraded version, you re-build the image in your build/dev environment, and re-launch the image into production. If you're using a volume, you get to use the same backing data.
It's rare that apt-get postinstall will affect any data that you would persist -- app-specific data you'd keep in the image/container, and mysql data for instance you don't really want apt touching anyway. But if a data migration is necessary you'd either manage it with a "utility container" (image that's designed to run a script then stop, rather than run and keep running).
two servers, one "active" serving the website, one "passive" ready to serve the website when you bring the first down and perform some switching (DNS or moving an IP address).
Load balancing two separate servers you can consider as active-active for the terminology.
Just because of how clearly you have explained it, I would surely want to buy your book when it releases. As of now, I know nothing about Docker except one or two buzz lines. And by reading your book when it releases, I am sure I can get a full picture. BTW, who will be target audience for your book?
Since the book is called "Complete Docker" it's broad in scope; I pitched the book to my editor as spanning beginner, intermediate, and production topics. I won't really go into advanced Docker in the book.
Right now the first several chapters serve as a "I'm a programmer but know nothing about Docker" guide, the mid of the book dives into Docker-specifics (how exactly volumes work, how Docker networking works, etc), and then it graduates into the Docker Ecosystem, dedicating some time to covering tools like Kubernetes, CoreOS, Fleet, Amazon ECS.
The latter portion of the book is a number of recipes: how to get a Ghost blog running behind an nginx proxy; how to launch WordPress with MariaDB; how to launch an ELK stack, etc.
Hopefully your book will mention in big bold Red letters why it's crucial that all the files except for data inside of a Docker image be OS packaged, every single one of them, especially the configuration files, and show how configuration management can be performed efficiently and at scale with OS packaging, as well as how one might perform change management with the whole lot.
And hopefully your book will mention SmartOS and Triton and zones providing full isolation for Docker.
I'd say: you should learn as much as you can! That way you can decide for yourself whether or not Kubernetes is for you.
One thing Kubernetes IS helpful with at small scales: portability. If you're fully Kubernetes/Docker, then you won't get locked into (eg) AWS's ecosystem. It's relatively easy to pick up an entire Kubernetes cluster and move it from AWS to DigitalOcean to private hardware.
So even if you're at small scale today but want to design for portability, I'd definitely look into Kubernetes.
Hrm not yet. I'll probably put a mailchimp form on my blog. If you use twitter follow me @bkanber; I tweet rarely but I will tweet about any mailing lists or impending book releases. Appreciate the interest!
The way I think about it: I remember in the 1990s when you needed to put up some web pages, you had to bring up an entire "web server", and that's all the server would do.
Now, think about it from a reductionistic engineering perspective. What do I really want this server to do? Well, it accepts TCP connections, parses a request to figure out which file (at the time, it was all files) to server, sticks an HTTP header on the file, and shoves it down the socket.
This task is so simple that a skilled network programmer can nowadays literally bash together a 1990s-level static HTTP server in an day, with nothing but a socket library and some basic string handling. (It may not be great and it probably is insecure, but, well... see also "1990s web server"....) The code to do this is perhaps in the dozens or hundreds of kilobytes.
But that's not what I had. I have a full computer that physically needs to live somewhere. It has hardware ethernet and hardware graphics cards and a physical monitor and a power supply and RAM and, basically, hardware hardware hardware, the failure of any one of which means the system is either difficult to change or outright down. I have an entire Windows operating system, which even in the 1990s was hundreds of megabytes of code, endless code. Code for a windowing system, for pete's sake. Code for the audio subsystem. Code for accessing the hard drive. Code for access code that accesses code. Code code code code code, a bug in any one of which means the system may be down or insecure. My website, which at the time was quite likely in single-digit megabytes in size, was a tiny directory lost in a sea of files on the hard drive.
Over the past 20 years, the commodity hardware world [1] has been slicing away at the fact that several dozen kilobytes of code are being accompanied by hundreds of megabytes of support and literal pounds of physical hardware. Hardware went first with VMs. VMs got lighter and lighter. Lightweight hypervisor solutions sliced away at the heaviness of the VM. Containers slice away at the OS. Things like Kubernates slice away at the idea of a container living somewhere physically.
We're trying to free that several dozen kilobytes of code to be just several dozen kilobytes of code, as flexible and easy-to-deploy as several dozen kilobytes should be, if you weren't mired in the world of hardware and OSes and code and strong physical connections.
(Data storage is more complicated, but in a lot of ways, the same principles are in play.)
Operationally, containers are very exciting. However, in terms of "magic technologies I don't understand", I don't think they're worth stressing about "getting old" or anything. It's mostly "just" a big pile of practical considerations in trying to not just build that world, but in some sense also undo decades of grinding-in of the physical world to our operational considerations. If you want worry about getting old and out-of-date, worry about that let you do something that you couldn't before, like GPU programming or deep learning.
[1]: Which must be specified because mainframes beat us all here decades ago, in a lot of ways.
^^^THIS is an absolutely great comment. I realise this isn't the major thrust of it, but it's the single best description of what containerisation is, and what problem it solves, that I've ever seen.
For the most part if your application is developed 12 factor style: http://12factor.net, you shouldn't have to worry about these things as a developer. Stuff like this is more in the ops space.
Cloud orchestration: the logic for how your platform provisions and launches instances (think ec2) into your fleet of resources.
Container orchestration: the logic for how your applications (containers) are distributed across your fleet of instances
Kubernetes: Software that runs on your fleet and manages how/where/when container orchestration happens. There are many players in this space such as mesosphere, swarm, ecs and so on.
I always think (thought?) containers are good if you own your own hardware. I use 'micro' cloud vm instances for my deployments and don't feel the need to squish more functionality into a $5/month VM.
all these answers are helpful but misguided. it is a lot of jargon but there is some value to be had, depending on your situation.
i'd argue that this situation is if you're a developer at a large company or enterprise. developing can be a nightmare because testing and/or deploying is so difficult. sometimes this is by design, sometimes by virtue of incompetence. other times people want to change it, they just don't know how.
tools like this claim they can change this. whether that's true or not. well, that depends on more than just what tools you use but it is part of the equation.
In case anyone's curious, I blogged about the backstory of the acquisition.[1] tl;dr: We at Joyent are elated, and we believe that this will be a huge win for our customers, for our technologies and for the communities that they serve!
Congratulations. I hope Samsung nurtures the technical excellence at Joyent.
Please, please continue to develop your public cloud offerings. Having options other that the myopic, me-too, feature-matching, monoculture that is AWS/GCE/Azure is incredibly important.
That said, for my use profile, you guys need to work on your price competitiveness. Hopefully Samsung will inject the necessary cash for economies-of-scale.
The Samsung TV I got came with a pirated copy of linux, which is quite a feat (came with a copy of the GPL 3, which is a clear breach of the GPL 2 only license of linux).
They are legally bound to contribute their modifications back to Linux (if they distribute them), it may be a different matter when they don't have to.
I can't say I know either way, however I do recall their Tizen SDK licensing not being accepted as open source due to it only granting the right to use certain components on 'Tizen Certified Platforms'.
It's the first sentence of the linked blog post, and the meaning seems pretty clear - instead of waffling on about how amazing an experience it's been and how much they've learned blah blah blah they just say "hey we got bought by Samsung."
We use Joyent for nodejs microservices, chiefly any server side DOM manipulations (using jsdom), and it's been solid! Currently evaluating docker container management and will give Triton a go...
I'm happy for you. I love your talks and your passion for good software that goes beyond feature checklists and the bare minimum to ship and be patched later. Have fun and fuck Oracle, the anti King Midas of companies.
Bryan - hope you see this since I don't appear to have your email anymore. Do you guys have a list of maximums for Manta anywhere? Specifically object count - but node count/total storage/etc. I'm guessing that was part of the samsung testing :)
Samsung is a typical asian electronics company (has a hardware focused history and very good at it, but doesn't understand or respect software). I'm so glad that node.js is not under Joyent control.
Having worked in a Samsung office for enough time to have an informed opinion, I can emphatically say that the "hardware focused history" has nothing to do with their lacking on the software front. They have some amazing engineers (especially at HQ in Korea), but the main obstacles are partly Korean culture in general (especially as it pertains to engineers) and partly the management practices of the company. The best things they develop (either in partnership or internally) usually never have a fair leg in the market. I'm certainly not apologizing for the company, just saying don't think the issues you see are directly the engineer's fault.
I don't remember the exact Korean wording, but a common phrase said by Samsung engineers (under their breath) in Korea basically translates to "I'll do what you say, but it's only because I'm at <name of a Japanese labor camp that I forget now>." My time there really made me appreciate how much better we have it here!
> They have some amazing engineers (especially at HQ in Korea), but the main obstacles are partly Korean culture in general (especially as it pertains to engineers) and partly the management practices of the company.
Can you explain the cultural aspects (and management shortcomings) in more detail for those of us who haven't experienced Korea? Thanks.
Haven't experienced a Korean workforce first hand, but as a Korean I can probably provide some insight into the culture.
In general, the culture is very hierarchical (respecting elders) thus seniority and age play a big factor in leadership. I would assume that this makes it very easy for mediocre (but good enough to survive) management to stay in positions far too long. Asian culture, but Koreans especially, can be masochistic1 and martyr2 like in their methods. In some ways this creates a tremendous amount of excellence, but it also creates soldiers who may not necessarily be incentivized to innovate, and those who innovate, out of cultural respect will tend to not dare "disrespect" seniority. Being a good soldier makes for career advancements.
Management can also claim the failures of the company as their own personal failures, rather than the gray collective failure of a group(e.g. japanese ceo's claiming company failure as their own), thus employees kind of get absolved of blame.
Work in Korea also tends to be a live at the office kind of affair. It's not uncommon for the white collar folk to go into work at 7a, and stay until 7-9p. Then go drinking with coworkers until midnight, and rinse and repeat. While this makes for very strong bonds amongst coworking peers and better teams, I would imagine that it also takes a toll on the mental capacity of the workforces.
Mainly, I wouldn't be surprised if work in Korea tends to be death marches all the time based on the martyrdome syndrome and masochistic syndrome for career advancement.
It's been a while since I was there but as a lowly US Army Private stationed there and working closely with Koreans old and young as an ambulance driver, there are a few things I still remember clearly.
Basic courtesy and being polite are pretty much universal, so try to be humble and keep an open mind and observe. Older people often are more experienced and savvy about how to handle something so being deferential to them is a no-brainer. You'll spot the incompetent ones soon enough and so continue being polite but ignoring their guidance.
One thing that drove me nuts but took a little while to figure out was that people would almost never say no to a request. I would ask if something was possible and they would pretty much always say yes, not wanting to disappoint, but I would have been better off hearing a 'no, sorry'. I wanted a realistic answer and theirs were often far too optimistic.
Me: "Mr Son, can we make it to Osan in 30 minutes?"
Him: "Yes" (always yes)
An hour later we show up late to another hospital with a very impatient air crew idling and waiting for our patient transfer. We would have all been better off hearing a "No, it takes an hour". Just one small example but this scenario repeated frequently.
I learned that when speaking to a group or doing a presentation it was customary to apologize for your inexperience on the subject and promise to do your best. Tim Berners-Lee could start a talk with "I'm sorry I don't know much about the World Wide Web..." for example and Westerners would laugh but the locals would be like "He's off to a good start".
Understanding this helped me get more realistic answers though. Even if I thought I knew an answer I would keep the question more open-ended to solicit a better response.
Me: "Mr Son, I'm sorry but I forget how long it takes to get to Osan, can you remember?" (said even after making the trip solo dozens of times in often difficult and stressful conditions).
Him: "Ah yes, there is new road construction halfway there so I'll teach you an alternate route to bypass it. We'll be there in about 40 minutes".
Had I not learned to ask a better question, I never would have learned his faster alternate route.
Korea was a lot of fun and I really enjoyed hanging out with the older Koreans who worked on our base, they were a constant source of good info once learned how to talk to them. I also had a 'ration card' entitling me to buy the occasional bottle of Crown Royal on base and give them, which always helped!
I only have secondhand knowledge (my brother worked at Samsung HQ for almost two years after grad school & a post-doc), but Samsung managers are often corrupt and treat their employees like trash. Foreigners have it a little better, they're allowed to go home at relatively more normal hours, but if you're from Korea and Korean, you do whatever the boss tells you or they retaliate hard since there are plenty of starry-eyed hardworking Koreans ready to take your place to work for the biggest company in & the pride of Korea. My brother wasn't completely immune either, they stiffed him on bonus pay by 50% on a max bonus year & lowballed him compared to all other foreigners by almost 2x compensation for similar credentials (our suspicion is because he's Korean - they refused to give him a reason. When he tried to go higher through a former Samsung VP relative of ours, who got a high ranking HR person to investigate, he got warned by his HR manager with an implicit threat), and he was a hotshot who finished his chemistry PhD at 25 at a top 25 US program who had his choice of jobs/academic positions. He ended up making it through by making a deal with his boss to leave him alone and my brother would avoid exposing corruption/bad decision making & getting him sacked (boss refused to teach or lend any resource for learning technical Korean chemistry terminology because his experience in American graduate school had him learning it on his own - it was explicit retaliation against western academia), and my brother still hated his life there (thought about suicide almost every day there); his first reaction setting foot on US soil again was "I love the taste of freedom!".
Whenever someone in HQ tells you to do something, you do it, and if not, you get punished, whether that's through decreased compensation or they fire you. You can be the engineer that tells managers that you can't draw parallel lines that intersect (metaphorically speaking), but they don't want to hear it unless the competitor proves them wrong with sales results - they are notorious for not innovating, but copying competitors. Samsung runs on waterfall from top to bottom. Oh, and if they tell you to do something early in the morning, you jump.
Samsung also tends to mainly promote Koreans - if you're not Korean and want to get ahead, you'd best look elsewhere because you're not likely to accomplish it. I hear this also applies to their American divisions as well, but I do not know anyone who has worked for them here. The family who owns the company has had some history with corruption in Korea as well, so this isn't too surprising.
More on the cultural side, Koreans can be very rude by western standards - it also is demanding on expectations. You're often expected to dress a certain way (this part is no different from most of the world, but disregarding norms is probably more harshly regarded in Korean culture than most), as well as join bosses & other co-workers going out to drink and doing karaoke at night. Deferring to father-like/older figures is built into the language with even how one addresses another, and that bleeds into how companies operate (& people treat each other - even being one month older can come into play unless you have leverage of some sorts). There are also built in opinions on processes, from parenting to schooling to work. Koreans tend to also be quite xenophobic/racist, even against Koreans who cannot speak Korean (maybe even moreso towards Koreans who cannot speak the language). Society sort of disowns those who are not immersed in the culture. Non-Asian foreigners are special, but they are excluded from much of anything of consequence (probably to avoid stirring the pot with other nations).
There's probably a lot more I'm missing here, but tl;dr if you aren't an extremely mentally tough person, working for a Korean company in Korea can crush you thoroughly. It is a different world, and people who do so from western society are left without a support base.
Just to provide another perspective, I'm an American anglo who has lived off and on for over 30 years in South Korea, speak the language near fluently, have worked in a variety of Korean institutions from a rural health center to a top research university. Along the way I have benefitted mightily from friendship with some great colleagues and much enjoy the rich culture and fascinating challenges of living in a rapidly changing society. Well-satisfied expats such as myself are not few in number. No surprise, the key is learning the language and using it for creative engagement with Korean society.
That said, I think it's fair to say that among many Koreans Samsung is viewed with respect but no affection. The company history dates back to the Japanese colonial period and is imbued with the pre-WWII Japanese zaibatsu mentality. What some posters are generalizing as "Korean" seems more the rather specialized local culture of Samsung. Personally, I would never work for them.
That's not a corporate thing, it's the culture of the entire country. Try not to condemn an entire culture just because you disagree with how they do things. From the perspective of Koreans, Westerners do lots of really unpleasant things as well.
There's also lots of very positive things about Korean culture, just like there are about Western cultures.
> Having worked in a Samsung office for enough time to have an informed opinion
That's Samsung proper. Not their incubator setup, which is different. Companies like Mapzen were bought by Samsung, but have a completely different culture.
I've heard that too, I remember a story from a worker who said they put all the Engineers into a room and said they can't leave until an issue was fixed. Took them two days.
I have no personal experience working for Samsung but know quite a few who did work for their US team and most don't have a positive experience. I also know some who worked for other South Korean companies. Lets just say the Korean work culture is quite different than in a typical US company. Hopefully the Joyent guys will be shielded from all this.
I worked with two project managers from their US team STA (Samsung Technologies America). One was OK. The other one was a fucking psychopath. I'm talking about needlessly and viciously backstabbing all in range. Totally unbelievable. Never seen anything like it, and god help me I hope I never see it again.
Later, I visited their HQ in Korea and met some of their Indian and Korean development teams. They were competent but the work culture was toxic. People were expected to attend 11PM daily review meetings where the issues list was painfully re-read pointlessly and nothing at all was achieved. Koreans were expected to live in numbered company compounds, catch a numbered company bus and ascend to a numbered floor in a numbered company skyscraper after an airport security style physical check. Smoking allowed strictly twice per day in a designated corner of the compound. All movement is tracked. Psychotic Confucian management style. I honestly thought to myself: if this is a vision of the future, I want no part of it.
Then you should see the japanese way of working, it's also quite fun! I believe this is common in most of asia. I have worked in China and it's not quite different at all from what you're saying.
Working in China is way different from Japan/Korea, at least for the companies I've been exposed to. There is much more freedom in how you get things done, there is less hierarchy, there is more value placed on programmers (which is why Koreans, Japanese, Taiwanese seem to come here to work in spite of the pollution).
My experience in Shanghai/Suzhou is very different fortunately. Not very long days if not needed, high salaries and quite a bit of freedom. Very Western feel while we work with Chinese companies with no Western employees.
From what I understand Samsung is a huge conglomerate of divisions that have little to do with each other.
Samsung Semiconductor is happy to supply chips to Apple even though they're the biggest rival of Samsung Telecommunications the builds the Galaxy phones. Conversely Samsung Telecommunications may or may not pick Samsung chips for their Galaxy phones, it doesn't really matter.
So I guess it depends to what division they will be attached. If they're attached directly to the group, they may actually keep complete freedom and just have a different owner.
Seems like the average complaints people have working at BigCo.
Compared to the kind of stuff that Microsoft, Apple have done in the past in the software world.
And I do not see how this is bad, maybe Samsung is trying to learn why 'Western' engineering is so much superior to 'Asian' engineering by this acquisition ?
Some of the rumors going on say that management back home is very provincial and are rigidly Confucian (hierarchical), which they enforce on their satellite operations. Also, your basic big corp in tech are quite internationalized from the start even at home; e.g. IBM, Microsoft, Intel, Facebook, Apple already employ those from many different cultures even in their home offices.
> And I do not see how this is bad, maybe Samsung is trying to learn why 'Western' engineering is so much superior to 'Asian' engineering by this acquisition ?
It depends if you want to be part of their learning experience or not! Chinese companies have adapted pretty well to acquisitions and satellites, so it isn't an Asian thing. Japanese companies have been known to have similar problems in the past and even currently (though it really depends on the company, and many have had a lot of time to gain experience).
Actually, Japan is another country that is notorious for not being as great as they should be at software, maybe there is a common thing going on?
Considering Samsung has been known to push for proprietary technology (they tried to create market of Android apps that would only work on Samsung's phones) I'm little worried there is going to be some sort of new node.js fork that's incompatible with current one and has some weird licensing stuff or something.
Microsoft is a software company. Samsung is not. If you think they are, then you probably haven't used one of their bloated, buggy phones that they stop updating way too soon.
What it has to do with it being Asian is the culture of conformity and group think; this is in stark contrast to Bryan's statement that at Joyent everyone leads and is led by taking individual initiative.
Don't know. There isn't even a word about Node in the statement from then CEO, but before today I didn't even know what the main business of Joyent is so I can't say how they and Samsung think it's important for them. Just figure, my first thought was "Samsung wants Node, but why?" :-)
Anyway, it's not tightly bound with Joyent anymore so even a worst case scenario won't affect ot much.
How do you come to the conclusion that the entirety of Samsung don't understand or respect software, when they clearly show great contributions to OSS? [0].
Well, in the case of Linux they are legally bound to provide the source code to any modifications they ship in binary form to a third party due to the GPL, I don't know what their track record is when they don't have to release the source code.
Yeah, this comment is ignorance at its finest. Mapzen, for example, was also acquired by Samsung, and it still operates as its own little startup, just with more financial security. Joyent seems to be going the same way. They will remain Joyent, but with support from Samsung.
There's a reason Samsung seems to be not that great as a software shop. I wrote up a long post on HN almost 2 years ago (God, it's been 2 years!!). I'm reposting the link here.
Hey, maybe Joyent can finally afford to refund the people who supported it when it was textdrive and then went back on their word!
Snarky I know but I'm still bitter about how it was all handled..
"How long is it good for? As long as we exist." As a lifetime plan purchaser, I wouldn't say no to a new Samsung fridge... Although, I think we're just beating the dead horse for old time sake—which in this case is fun in itself. Though, it would be kinda awesome to hear a Samsung exec weigh in on the matter.
As a fellow early textdrive supporter, I'm pretty happy with the value I got out of the deal. I'm still using my free strongspace account 10ish years later. I think there was some email about a free hosting account I could theoretically use with some company, but I never looked into it. Honestly, textdrive's hosting was never really of a quality I could use in a professional context anyway. I understand some bitterness, but any "lifetime" deal has to be taken with a pretty big grain of salt, and I'm pretty happy about what I got for the money.
> I think there was some email about a free hosting account I could theoretically use with some company, but I never looked into it.
We were offered either one or three free years of hosting on a Joyent SmartMachine, depending on what your initial investment was. After this free trial, you would be converted to a regular customer.
Adding my voice to the bitter-parade. Joyent wouldn't be where they are today without us early bootstrap investors. It was basically a Kickstarter campaign before Kickstarter existed. Not only do I feel like I failed to receive my early investor benefits, but WORSE off was how the whole situation was handled. We had to beg and plead to get our data back. The company essentially went dark to all of their oldest, most loyal customers.
It wasn't about the money or the value I got out of it.
For me, it was the way I felt like I was treated.
It felt like, hey, we've pivoted the company but we still have to support these old users and they are becoming a pain, let's give them a credit on something that's kinda like what they paid for and be done with them.
Then when enough people complained, then they tried to find some solution at the last minute and even that felt dubious.
Let's try to cobble something together that's barely working for these people and make it someone else's problem.
I totally take the "lifetime" deal with a grain of salt but it wasn't like the company folded because it ran out of money or wasn't successful. It was more like we got successful so we forgot about the little people.
Although perhaps Samsung's acquisition might continue with the trend, improving Joyent's bottom line by ignoring all previous financial obligations that may have existed before the purchase...
Hats off Samsung. You have just acquired a truly world class Engineering team.
Congrats to all my former colleagues who are absolutely amazing at their jobs and wonderful people to work with. Samsung looks like a very good match. Hope the transition goes well.
Forgive my ignorance but can someone explain how Joyent's acquisition moves Samsung towards their strategic objectives? In other words, how are they going to exploit this technology (and brain gain)?
From the CTO's blog post, it seems to me that Samsung became interested in Joyent not for its most famous product, Node.js, but for its new product, Manta, which is some sort of serverless, distributed, data-storage and processing service (https://www.joyent.com/manta). Manta might be well-suited for the new hotness, the Internet of Things.
So that's how I saw it progressing. Samsung was looking into how to get in on the Internet of Things, stumbled across Manta, and it went from there.
Manta seems like they are throwing the kitchen sink at those problems, tons of containers, proprietary OS, proxies, etc.
If you want to store objects in a distributed way, why not just use something like RethinkDB, which has proxying/sharding etc. built in? RethinkDB even has a way to execute code with 'js'.
Or for serverless processing, why not AWS lambda or the crop of clones?
Manta doesn't run on a proprietary OS, but runs on SmartOS, a distribution of illumos [1][2]. In fact, almost everything at Joyent is open source, including all of Manta [3]. As for why not anything else, Manta has been around since 2013 [4], before other similar solutions, and allows you to run arbitrary programs on your data. As an example, some of our customers run ffmpeg on videos that they upload, to produce different variants to then store in Manta.
For the record, there is no "proprietary [operating system]" involved here. The Triton stack (upon which Manta is based) runs on top of SmartOS[1], a wholly open source operating system built around an illumos[2] core.
Samsung generates a lot of sensor/IoT type of data (e.g. smart phones, smart TVs, medical, industrial). They'll need the infrastructure to process and store all these data.
I have the same reaction, for me samsung is more a hardware company and the software they make is only to support their hardware. So is all future Joyent software development will be close sourced and only used by Samsung?
I always thought Microsoft/Google or the likes would acquire Joyent as it is good product fit. Microsoft chose linkedin instead :)
For Samsung this is all about IoT .Samsung wants to own both the devices and backend. A good move IMHO
MS already has Azure, with an infrastructure built on their windows core... Not to mention having more services... and while Joyent's Docker story is really nice, MS seems to be heading in a very similar direction with the Windows Subsystem for Linux as Joyent took with SmartOS/Solaris, it's a poor tech fit.
I'm not sure where this will take things with Samsung, likely backend services to support their own "store" on their devices separate from Android. If it's internalized, I only hope they release more of their infrastracture tooling, which is very cool to say the least.
There is a world of difference between a new, untested implementation of Linux system call translation to a _very_ different type of Kernel on a desktop class operating system than the version on Illumos. The best thing Microsoft could do with their cloud is burn it to the ground and start again with Joyent's tech.
Joyent's tech is very good, don't me wrong, but I think this is a bit harsh.
- lx-branded zones and window's new layer are both doing system call translation---you're post doesn't contradict this but doesn't make it clear either.
- At some point lx-branded zones where equally untested.
- "Desktop class" is misleading because NT was designed for servers. Now yes Windows as a whole is hamstrung by a business model putting desktops and backwards-comparability first, but by definition the Linux layer can ignore a lot of Win32 precedent.
- Yes Illumos is Unix and NT isn't, but the difficulty in translating linux systems calls lies not with the ones backing posix-interfaces, but the Linux-specific ones. (Joyent devs can give you some great horror stories here.) It may be a bit harder with NT but it was hard already.
The work on desktop Windows and Linux integration is mainly for developer experience and making Windows attractive for non .net devs. Doubtful that it'll have an impact on Azure unless it becomes extremely mature quickly or if that was MS's intentions at all.
I'm ignorant on why Joyent is so superior to Azure. Can you give some reasons? As an outsider, Joyent looks like a small company with unproven (large, successful customers are the main criteria) products that cost more than the comparable solution from AWS, GC, Azure.
> Joyent as it is good product fit. Microsoft chose linkedin instead
It's a good product fit, but the fit is so good that Microsoft basically built a more-complete competitor in the form of Azure. They built Joyent instead of buying it.
LinkedIn was different because Microsoft could never build LinkedIn itself.
Microsoft has good PaaS story when it comes to .NET stack. Especially product Azure App Service[1].All though it can technically host nodejs,java etc ,it still is based on IIS and windows stack. Joyent has more native offering for nodejs which could have been Paas for nodejs(Azure App Service for nodejs )
As Microsoft becomes the new IBM, except for XBox, their future revenue is firmly from the Global 10000 Enterprise space. LinkedIn gives them the entire database (up to date! SF.com doesn't give you that!) of decision makers in their target market and their web of professional relationships.
edit: I should have been more specific.. when an open source project is under a company's wing and it gets acquired, you don't know what can happen, even under MIT. Look at express recently. Since it's the under the Node Foundation now, this is not a big deal. Had it happened a short time ago, there may have been further turmoil in the community.
Given that Node.js has always been MIT licensed, that's impossible. That's one reason why it had a liberal license; just listen to Cantrill's Oracle rants sometime to get a better idea.
Cantrill advocated the fact that CDDL was copyleft as a good thing[1] (it's a bad license overall, but the point is the same) specifically mentioning that the fact that Oracle couldn't use new DTrace and ZFS improvements from the community was a good thing. I agree (though I much prefer the GPL).
So actually all of Cantrill's worries with Oracle exist with Node.js's license. I am sad when I see large free software projects licensed under MIT, we are doing ourselves a disservice by allowing proprietary software to take advantage of the free software we've developed.
[1] It's part of his Fork Yeah! talk about the fork that is illumos.
There are people out there like myself who are convinced that GPL is the most vile, fascist license out there.
On the other hand, I have nothing against CDDL. I think the incompatibility of CDDL with the GPL is what saved the illumos community (even if inadvertently) from having their technology cannibalized by GNU and Linux. I'm happy that BSD is benefitting from illumos (and vice versa), but I'm even more happy that Linux isn't. <--- I do not represent anyone but myself, the position expressed here is my own and not of anyone else.
> There are people out there like myself who are convinced that GPL is the most vile, fascist license out there.
Not proprietary software? Not "open source" licenses that don't allow modification (which is also proprietary)? The GPL? The thing that gave us free software in the modern world? Well, that's certainly one opinon.
> On the other hand, I have nothing against CDDL.
The CDDL has a provision that if any part is held unenforcible it will be modified to make it enforceable. As a matter of licensing, that's incredibly devious and bad. There's other super dodgy parts of the CDDL that make it a bad license purely from a license point of view, let alone from "freedoms given to users" perspective.
> I think the incompatibility of CDDL with the GPL is what saved the illumos community (even if inadvertently) from having their technology cannibalized by GNU and Linux.
What are you talking about? Why do you care if GNU/Linux takes your code and modifies it? No damage is done to the original.
> I'm happy that BSD is benefitting from illumos (and vice versa)
Actually, because CDDL is somewhat-but-not-really copyleft, many BSDs essentially have copyleft requirements if you enable ZFS or DTrace. Which is funny, given how much they go on about "freedom to make proprietary software" (something I'm against).
> I do not represent anyone but myself, the position expressed here is my own and not of anyone else.
>What are you talking about? Why do you care if GNU/Linux takes your code and modifies it? No damage is done to the original.
This is more about the fact that when they take it it's worse for the project than a proprietary fork. When BSD/MIT/(other permissively licensed code) is used, modified, and re-licensed to GPL, the permissive project can't take those improvements without also re-licensing. The CDDL indeed saved illumos from being cannibalized and disappearing as a community. Had its features been easily ported to Linux without a licencing problem, the project would probably not have survived, simply due to the fact that there would have been few reasons to stick with it. It's not just about what code is out there, and you should know that.
>Actually, because CDDL is somewhat-but-not-really copyleft, many BSDs essentially have copyleft requirements if you enable ZFS or DTrace. Which is funny, given how much they go on about "freedom to make proprietary software" (something I'm against).
Only in regard to changes to DTrace and ZFS. Still totally fine to build proprietary versions of BSDs that include those pieces.
I won't go so far as the grandparent post here, but I'm one of those developers that couldn't give half a care about "user freedom". It's nice to have source as a developer, but my motivation has never been "freedom", insofar as using and understanding the software, so I understand where he's coming from in his complaints about the GPL. As a developer, it's more constraining than the permissive licenses out there. To some, that's a good thing.
> The CDDL has a provision that if any part is held unenforcible it will be modified to make it enforceable. As a matter of licensing, that's incredibly devious and bad. There's other super dodgy parts of the CDDL that make it a bad license purely from a license point of view, let alone from "freedoms given to users" perspective.
Wait until you discover the BSD license, which lets you do whatever you want with the code... you might get a stroke (:-D)
GPL gave me no freedoms, it gave me the GNU userland tools and GNU/Linux (illumos / SmartOS and AT&T SVR4 tools are just fine, thank you, do not want GNU!)
Luckily I don't have to license anything under the GPL, GPL cannot take that away from me as long as I don't link with the kernel code licensed under it, or derive code / work from it.
And I'll be extra careful I don't ever land myself in a position where I would have to license something under the GPL. Any other FOSS license is better than the GPL, I have no problem with open sourcing, and cannot wait to clean up my code in order to set it free.
But not under the GPL license.
Finally, CDDL is both FSF and OSI approved: the open source initiative looked over the CDDL license and deemed it to be in compliance with open source. I will take what is in my view their expert finding over your position, but thank you for sharing your opinion.
> The CDDL has a provision that if any part is held unenforcible it will be modified to make it enforceable. As a matter of licensing, that's incredibly devious and bad. There's other super dodgy parts of the CDDL that make it a bad license purely from a license point of view, let alone from "freedoms given to users" perspective.
No, that's a pretty standard severability clause that helps the user. It means that if one section of the CDDL is found invalid, users can still use the software. The GPL on the other hand, by refusing to have a severability clause, means that if any part of the GPL, no matter how inconsequential, is found invalid, the entire license is null and void and no one can use it. That intentional omission makes the GPL a much more user-hostile license than it should be.
You know fascism is a political/economic system that advocates a nationalistic war focused culture and foreign policy, a military aristocracy and sort of mid-level command economy right? Which is to say literally nothing like any possible outcome of the GPL.
What does the license have to do with anything? Joyent themselves are now behind SmartOS, which is based on Solaris and its CDDL license. An open source license being more free or more liberal has absolutely nothing with the ability of the community to fork and/or continue support if the original owner discontinues it.
It's already been successfully forked once, under 'io.js', when a group of developers were frustrated at the slow pace of Joyent's custodianship. io.js came back to node.js after politicking.
Same initial reaction. I can't stand Samsung and refuse to have anything to do with them. All started when they took my 5k dollar tv and started injecting ads and such all while claiming they didn't (this was before the recent media caught on)
I'm sorry but how is this a good fit? What is the synergy here? This particular sentence is incredibly vapid:
"By bringing these two companies together we are creating the opportunity to develop and bring to market vertically integrated mobile and IoT services and solutions that deliver extraordinary simplicity and value to our customers."
As I toil away on a node based project that is interfacing with Samsung's Artik platform (both the Artik 10 board and Artik Cloud) I finally decided to call it a night -- check HN real quick -- and discover that Samsung is buying the original stewards of node. I almost thought the lack of sleep was putting me into psychosis...
Well, buffering 4K video, while running other active services in the background... finally broke down and installed FB messenger again, and it's definitely better, but there are a lot of consistent apps that tend to be very bloated as it is.
Given how well React Native tends to work, and I have to be honest, I like the ecosystem structure better than most apps I've worked on in general, I can see it actually working out better in many cases.
> finally broke down and installed FB messenger again,
Prob not great for more advanced messenger functionality but Face Slim (https://github.com/indywidualny/FaceSlim) does notifications and simple/group chats fairly well.
What sucks is I was perfectly happy with fb web, until they started cutting features in mobile (can't post to a friend's wall, can't do a lot of things now). The biggest issue is FB the app didn't respect the environment/usability settings for extra large fonts.
The only thing JS has going for it is it's wide adoption. Basing a new language on it to run on low end devices just seems like a dead-end for me. At least the first link is to a typed subset of JS.
As a Korean American who also has worked at Samsung headquarters, I think it's more of bad news than good news, no matter how Joyent wants to spin it.
Its corporate culture only allows the most cunning, politically savvy person to stay alive and move up the rank, and thus most executives (all if I limit it to small sample of executives I've personally met) fit that model.
And shit literally flows downwards, where goals/promises set by them would be pushed downwards and engineers have to take the burden.
It doesn't help that Korean society is very hierarchal and based on Confucius principles, where you don't usually challenge older persons and/or someone higher in the rank. This is one example that describes serious problem - http://thediplomat.com/2013/07/asiana-airlines-crash-a-cockp....
For those of you who are intrigued and have time, I suggest watching Misaeng with English subtitles (https://www.viki.com/tv/20812c-incomplete-life). Samsung isn't as bad, but the same hierarchy, verbal abuse, social dynamics, and strict rules on paper format exist.
The best outcome would be if they leave Joyent's management and culture alone. But I doubt it.
I also have the first-hand experience of their applying the same "consumer electronics" mentality to completely different business which required high-touch sales.
There is no denying success of Samsung - multi-billion, international corporation. However, Samsung is only good at generating quality hardware products at mass scale. There have not been success in any sort of software and services. Perhaps they are trying to expand beyond their strengths, and I applaud that effort and they actually do need it, since it's only matter of time Chinese companies will catch up and produce as quality products as Samsung, as Samsung did to Sony. I hope it bears fruits. I hope they can allow Joyent to succeed and thrive, and learn from that.
Joyent funded most of node.js' development since 2010. Once node became popular and competing companies appeared, with the commensurate agitation and political fallout, Joyent became less involved with it.
Best wishes to all at Joyent. Thanks for all your work in the community over the last few years, particularly with node.js. Hopefully Samsung will give you the resources and reach to go on to better things in future.
So this is the moment where everyone with a sufficient device / install marketshare decides they need to buy cloud expertise?
"Samsung will immediately benefit from having direct access to Joyent’s technology, leadership and talent. Likewise, Joyent will be able to take advantage of Samsung’s scale of business, global footprint, financial muscle and its brand power."
Samsung is (and has been) terrified that their handset business will be slaughtered in the race to the bottom of the hardware industry. They tried to create their own OS to give themselves an edge over other Android manufacturers.
Their fears have come true (and now it's even happening to Apple). Hardware is an incredibly difficult, risky business, and differentiation is now in the services/software arena. That's why Samsung, in particular, needs this acquisition (and others like it).
One thing that I never understood was how the Android hardware ecosystem never saw contributing resources to developing more efficiently abstracted core Linux + Android support for their hardware compatibility challenges as a competitive advantage.
Disclaimer: I'm not involved in that space, so maybe they've started?
Samsung is in a good position to utilize Joyent in Asian markets, better than most cloud providers, and given the "smart" features of most of their devices, and how much they probably feel they're leaving money on the table compared to even Android's Play Store, it makes sense... doesn't mean it'll work... but even if it only applies to Asia and Africa, where they leave the more firm Android in place elsewhere, it could be a significant boost.
I don't expect to see Samsung take on GCE, Azure or AWS directly though... and would be really unsure if I were a Joyent customer right now.
I sure hope so, but the high performance nature and intent of SmartOS is at odds with the low electricity consumption, low number crunching performance of the Acorn RISC Machines processor. SmartOS is optimized for CPU bursting and squeezing as many tenants at bare metal speed as possible, while ARM is designed not to consume lots of power at the expense of raw performance.
That said, since SmartOS is based on illumos, and illumos is designed from the ground up to be portable, running SmartOS on ARM would be cool, since it would open the doors for illumos on embedded devices.
I would so love me an illumos based, iPad Pro-like tablet...
Android on servers, like Linux? SmartOS on Arm servers would probably make a lot of sense, given that Samsung might then have a vertically integrated infrastructure: mainboard/chipset, cpu, ram, ssd and software all made by Samsung for Samsung?
If anyone has a good background, what exactly does Joyent have that is so valuable?
It seems that NodeJS has moved out of Joyent. They have hosted container support that seems to run on solaris, which seems interesting, but a bit too much of buzz-wordy from their website.
I am not very familiar with this, so will be great if someone can explain a little bit. I read the comments around orchestration, but am more interested in Joyent's value proposition.
Well that sucks. They had a good run, but Samsung's business culture is probably going to ruin them. Just check out some of the stories of their internal software engineering process.
wow, Joyent. Took me a bit to remember what they were doing in the early early days since we're multiple generations or pivots or focus-shifts on now......but it was Textdrive/Textile/Textpattern CMS. Ha. Different times. At least some of that still out there in OSS Land
I am developer, but mostly work on desktop apps, or embedded devices or lataly on some MVC applications. But reading things like
"A Container-Native Stack for Modern Applications and Operations Increase development velocity while simplifying operations."
I have no idea what should I imagine and what is it good for...
any good introduction or explanation into what is it they actually do?