01. Pioneers. with Kelly Goetsch

‍Piotr Karwatka: Today my guest is Kelly Goetch, Chief Product Officer of commercetools - the headless e-commerce platform for the future. I'm really excited we can talk on how the e-commerce technology landscape is evolving and why your next e-commerce platform is probably headless. Hello Kelly! First of all thanks for accepting my invitation!

Kelly Goetch: Thanks for inviting me! Glad to be here today.

Piotr Karwatka: I first had a chance to know you around 2016/17 when you published your book “Microservices for modern e-commerce”. I guess it wasn't your first book?

Kelly Goetch: No, there was a summer there at Oracle, I was kind of bored to be honest with you, because we had launched a product but hadn't yet seen a lot of uptick. This was exologic and... You know it was fine, I was keeping busy but I was intellectually bored and I'd always wanted to write a book and I ended up pitching to O'reilly and they accepted the proposal and yeah, I wrote it, I took off a bunch of time and I sat down and I wrote it. That was a really good experience for me.

Piotr Karwatka: What I really liked in this Microservices book is that you started from the organization point of view. It's something pretty unusual because usually microservices are about some Java, Javascript whatever technology you have in there and you started with the organization. The microservices are based on the new team structures, the whole organization must adopt... Do you think that this team structure is an important success factor in implementing the service based architecture?

Kelly Goetch: I think it's the important success factor. I’ve spent a lot of time in really big companies like Walmart and Oracle, and these are each 100 000 person plus companies It was remarkable just how how layered the organizations were so I don't know how philosophical you want me to get here but you know, back in the day if you wanted to set up a database you had to have a team of very highly skilled DBAs to do that. Then you would build a DBA group and they would own the database and they didn't want pesky developers like me touching their pretty database. That’s how it used to be... You know if I was at a bigger company and I wanted to get a column added to a database table right like that was a really easy thing for me to do personally. If I had access to the database that's a five second change. But in a big enterprise you have to go through a whole change request process, you have to fill out a whole bunch of paperwork and then they take the request, they implement it in a development environment and then they give you access to the environment and then of course you know your your access never actually works  it takes and then you have to verify it and they have some stupid system that they use to manage their work where you have to market as done in the workflow yeah and then they promote it to QA and then they promote it to production and that's why you end up having all of these project managers, because all you're doing is coordinating the work across horizontal teams. To take a feature from a middle tier developer, adding a you know, maybe I want to start capturing the customer's birth date. If I had full access to that whole stack that's a five minute change, right? I go to the database I have the table, I go to the ORM, I expose that property, I go to my mid tier code where you capture the registration information, I add it there to the form handler whatever it is and then I go to multiple UIs and I just simply add it. That’s really easy if you had access to the whole stack. But you don't have access in those big environments and that could take 6, 9 or 12 months to make a stupid change like that. That was really frustrating to me because as a developer I don't want to get stuck in these big workflows, I want to get sh*t done, right? I want to work because that's what I do... And with microservices you're building teams that are focusing on a business domain and then you're allowing them to use technology to solve those specific problems. With cloud today you can do that because now you don't need a team of DPAs, you don't need a team of network admins, you can just go to public cloud. It's really easy if I'm a team of seven people and I'm tasked with inventory for the whole company or I'm tasked with a customer profile to just make whatever changes I want in my stack and then to the world I expose my single rest API. That makes it really easy because now I have a front door basically from what I'm doing and then everything else is an implementation detail. So I think everything stems from the organization and actually Melvin Conway in 1958 came up with... it was later coined Conway's Law but it's that the software reflects the organization structure that produced it. The old joke back then was if you have seven teams working on a compiler you'll end up with a seven pass compiler because…. you know... 

‍

Piotr Karwatka: Yeah, that's a classic right? 1958! So still... you know, still so true! You already answered the question I had a little bit. The question was: “what was the microservice back then in 2011 and how did the e-commerce look like back then, I mean the e-commerce project, the implementation project, how did they look back then?


Kelly Goetch: Yeah, there weren't really microservices, it was really 2015 when we started microservices at Oracle. But 2011 it didn't really exist as a product, as a company. I mean docker really wasn't even a thing you know back then not that you need a docker but you know that just kind of shows the state of technology back then is we didn't even really have containers. Microservices really started in the mid 2000s in Amazon and that was actually a mandate from Jeff Bezos where he said every service must have a well-defined interface, the technology doesn't matter, it must be externalizable meaning it must be able to be consumed by third parties. He had a whole manifesto he published and iterated on. I think that's really where modern microservices came from and then later in 2013 roughly is when the name “microservices” actually started to be used. That's when books started to come out you know, that was when it really became a thing. And even to today, with the exception of commercetools, it's still very much frozen in the late 90s and early 2000s. So you have single, large Java EE applications, the source code is there, it's packaged up in jar files, you have extensions that you put on top of it, it's connected to a relational database and then you have to deploy and manage that whole stack. And they have these cloud offerings you know, quote unquote, but they're not cloud, it's just hosted. It's taking an application and then hosting it for you. For a while at commercetools we actually had a marketing campaign called: “Is your commerce platform fake new?” and you know, that was a pretty successful campaign from what I've heard and it really stuck with people. It was a different story back then.

Piotr Karwatka: Maybe we define what's the definition for you, for the microservices. Service giving you date and time? Because this kind of example you can find in the books on microservices.

Kelly Goetch: I'd say it's small teams, roughly five to ten people, give or take. I think seven is about the optimal team size. They are cross-functional. There are people who do middle tier development, who do back end operations work, you have a product manager and they're all working together as a single team to solve a specific granular business problem into the world that they expose typically a rest API. They also have access and complete ownership of their data, of their database so in other words you can't bypass their API and read from their database. The only way you get things out or in is through their well-defined APIs. Then you know, a few other things, if you can build and release a microservice independently, I think those are also two hallmarks of actual microservices. Because you know, you don't want to have 50 microservices but then I'll have to coordinate on when you build and release right because that's not microservices that's just, I don't know what you'd call that but it wouldn't be good.

Piotr Karwatka: I think that this separation of concerns, lose coupling, the way you can deploy the services is what defines them. For me that's it, it’s one of the most important features. That they can be developed on different paces regarding the team requirements, the user requirements.
Let's let's go into migrations upgrading to microservices architecture, the topic which is I guess pretty important for our listeners because if you are a successful e-commerce today it means you started some years ago and you have reasons to migrate. Sometimes when I spoke to CTOs of the companies that started the e-commerce journey in early 2000's, so the the times when ATG was the e-commerce “thing” I guess, they were saying something like: “we started where there was like one or two platforms to pick, IBM, ATG so we had to build a lot of systems around those platforms” and it was 10 or 15 years of history, of building those systems, exchanging XMLs, running raw SQLs queries on bare material, mainframe like servers... And now the architecture is a total mess. It's 50 plus different services and we need to upgrade, streamline this architecture somehow... Do you think that this SOA - Service Oriented Architecture is a way to do this upgrade gradually like why do they why do they pick this way like going headless, going microservices?

Kelly Goetch: There’s SO in terms of concept and there's SO in terms of practice. In practice SO generally fails pretty spectacularly. I'm not a fan of SO. The whole point of SO was almost like a precursor to microservices and the thought there is: if you have all of these different APIs, whether they're REST APIs or SOAP APISs or in databases or wherever they are, the whole concept was you'd go to a a service bus and you'd plug them all in and then you can access whatever you want from that bus, which makes sense philosophically. The challenge you end up with is that thing, that SOA layer, then becomes very, very tightly coupled to all of the services on which it relies. It's a tight coupling, it's not a loose coupling and you end up having to change the CRM, the ERP, the OMS, the commerce platform and then the SOA system all at the exact same time because they all have so many complex interdependencies. That becomes really difficult. You end up with basically a monolith in your pipes. That was always the problem with SOA. Let me see if I get this quote right, I don't know who came up with it but I've always liked it. It's: “with SOA you have dumb endpoints and smart pipes, with microservices you have smart endpoints and dump pipes”. I think that's what we want with microservices, we want the pipes to be plain olds REST, HTTP web services you know, just restful web services. We want them to be dumb as possible. And then have the intelligence all be in the individual endpoints, so it's the complete opposite you know, it's all of... There's transformation logic and all sorts of business logic that you put in the pipes and I think that leads to a lot of coupling which leads to slowdowns which leads to spaghetti code that takes decades to unravel.

Piotr Karwatka: That's actually interesting, it's still pretty monolithic, right? It's kind of fussate with the services over one or many different platforms but still very coupled together...

Kelly Goetch: You know, it's the equivalent of welding together a bunch of big legacy applications and the very last thing you want is for those to be welded together. You want them to evolve independently and not have all those complex interdependencies.

Piotr Karwatka: I think that we have also the second segment of the companies and CTOs thinking on the migration towards headless architecture microservices and I guess that they are the guys who started a little bit later, around 2008/2010, maybe even later. Picking some, I name it “all-inclusive platforms”, like Hybris, Magento or some others. The platforms that have all the features you know, the CMS, OMS, PIM, everything with the front-end, everything included, so you can just run it out of the box, in theory right, in practice is totally different but this is how they get into this situation. And why are they getting out of this situation, so what are the reasons they are thinking of headless? What's your experience of talking with those guys, like with the CTOs?

‍

Kelly Goetch: Until very recently until us basically all of the commerce platforms were very suite focused. That was because back when you were just getting online for the very first time right, then this was you know the early 2000s, typically the 2000s. If you were a shoe retailer and you had 500 shoe stores and you wanted to start selling online- you needed a platform. You didn't have a lot of maturity and you needed a platform that did a little bit of everything. If you look at ATG, if you look at Hybris, if you look at IBM websphere commerce you know Demandware, Magento all of those products they had lots of functionalities which at the time was absolutely necessary. They had very lightweight CMSs so they did the front-end, they had an OMS, they had inventory management, they had all of these things. They had little things like a b-testing which today is crazy because that's all custom, that's a third-party app now you never rely on that in a suite so what they did is they they implemented basically these products uh as intended which is you know with a head on them hybris has a whole management layer for UI and i think what they realized is that over time there are a lot of websites started to look the exact same and touch points started to look the same and to differentiate yourselves. In today's market, if you're not selling a commodity good you have to differentiate on experience. Right? People just don't go buy a you know five thousand dollar jacket you know from some luxury retailer because it has the best cashmere in it. They buy it because of the brand association because of the relationship to the brand because of imagery marketing right all of those things. And you can't do that if you have a really lightweight cms.You can't deliver that experience. You end up with cookie cutter websites.

‍

(...)

‍

‍

‍