Alle afleveringen
397 afleveringen, van nieuw naar oud
-
Alois Reitbauer on Cloud Native Application Delivery, Keptn, and Observability
28 oktober 2020 · 32 min
Beluister deze afleveringIn this podcast, Alois Reitbauer, VP, Chief Technical Strategist and Head of Innovation Lab, sat down with InfoQ podcast co-host Daniel Bryant. Topics discussed included: microservices vs functions; the go-micro and micro frameworks; and the evolution of PaaS and how the new M3O platform fits into the landscape. Lees meer
Listen to the podcast for more.
Curated transcript and more information on the https://bit.ly/31Ju5e4
Follow us on Facebook, Twitter, LinkedIn, Youtube: @InfoQ
Follow us on Instagram: @infoqdotcomStay informed on emerging trends, peer-validated early adoption of technologies, and architectural best practices. Subscribe to The Software Architects’ Newsletter: www.infoq.com/software-architects-newsletter/
-
KIP-500: Removing the Dependency of Zookeeper on Kafka
19 oktober 2020 · 32 min
Beluister deze afleveringToday on the InfoQ Podcast, Wes Reisz talks with two of the engineers currently working on removing the dependency of ZooKeeper in Kafka. ZooKeeper is used to maintain the metadata store required to operate Kafka. While ZooKeepers removal from Kafka will simplify the operational complexity and improve some of the scalability aspects of the platform, it is a huge undertaking that represents major changes to the overall architecture. Justin Gustafson and Colin McCabe are two engineers working on change. On the podcast, the three discuss why the team made this decision, what the ramifications are, and explore what both the near and future state will be with upgrading and operating Kafka. Lees meer
Listen to the podcast for more.
Curated transcript and more information on the https://bit.ly/2H5w3hN
Follow us on Facebook, Twitter, LinkedIn, Youtube: @InfoQ
Follow us on Instagram: @infoqdotcomStay informed on emerging trends, peer-validated early adoption of technologies, and architectural best practices. Subscribe to The Software Architects’ Newsletter: www.infoq.com/software-architects-newsletter/
-
Asim Aslam on Microservices, go-micro, and PaaS 3.0
12 oktober 2020 · 25 min
Beluister deze afleveringIn this podcast, Asim Aslam, founder and CEO of Micro, sat down with InfoQ podcast co-host Daniel Bryant. Topics discussed included: microservices vs functions; the go-micro and micro frameworks; and the evolution of PaaS and how the new M3O platform fits into the landscape. Lees meer
Listen to the podcast for more.
Curated transcript and more information on the https://bit.ly/2GOyQLC
Follow us on Facebook, Twitter, LinkedIn, Youtube: @InfoQ
Follow us on Instagram: @infoqdotcomStay informed on emerging trends, peer-validated early adoption of technologies, and architectural best practices. Subscribe to The Software Architects’ Newsletter: www.infoq.com/software-architects-newsletter/
-
Anne Currie Discusses Cloud Providers and the Environmental Impact of Software
5 oktober 2020 · 27 min
Beluister deze afleveringCharles Humble talks to Anne Currie from Container Solutions, exploring the environmental impact of technology. They look at how technology compares to other industries such as aviation and farming, how the big cloud providers compare in terms of their commitments to reducing carbon emissions, and the impact of the choices made by individual developers and software architects. Lees meer
Listen to the podcast for more.
Curated transcript and more information on the website: https://bit.ly/30yU3jL
Follow us on Facebook, Twitter, LinkedIn, Youtube: @InfoQ
Follow us on Instagram: @infoqdotcomStay informed on emerging trends, peer-validated early adoption of technologies, and architectural best practices. Subscribe to The Software Architects’ Newsletter: www.infoq.com/software-architects-newsletter/
-
Matthew Skelton and Manuel Pais on Team Topologies
28 september 2020 · 32 min
Beluister deze afleveringIn this podcast, Matthew Skelton and Manuel Pais, co-authors of the book Team Topologies, sat down with InfoQ podcast co-host Daniel Bryant. Topics discussed included: the role of a modern software architect, how team design impacts software architecture, creating “team APIs” in order to reduce cognitive load, and the benefits of building a “thinnest viable platform”. Lees meer
Listen to the podcast for more.
Curated transcript and more information on the https://bit.ly/3iyapiT
Follow us on Facebook, Twitter, LinkedIn, Youtube: @InfoQ
Follow us on Instagram: @infoqdotcomStay informed on emerging trends, peer-validated early adoption of technologies, and architectural best practices. Subscribe to The Software Architects’ Newsletter: www.infoq.com/software-architects-newsletter/
-
Pat Helland on Software Architecture and Urban Planning
21 september 2020 · 27 min
Beluister deze afleveringToday on The InfoQ Podcast, Wes Reisz talks to Pat Helland about the relationship between software architecture and urban planning. Pat explores planning for future growth, regulations/standards, and communication practices that cities--and software architecture--had to evolve to use. He uses these comparisons to distil lessons that architects can use in building distributed systems. A key theme throughout the podcast is constraints improve system design by restraining project scope. Lees meer
Listen to the podcast for more.
Curated transcript and more information on the https://bit.ly/33IFR8A
Follow us on Facebook, Twitter, LinkedIn, Youtube: @InfoQ
Follow us on Instagram: @infoqdotcomStay informed on emerging trends, peer-validated early adoption of technologies, and architectural best practices. Subscribe to The Software Architects’ Newsletter: www.infoq.com/software-architects-newsletter/
-
John DesJardins on In-Memory Data Grids, Stream Processing, and App Modernization
14 september 2020 · 33 min
Beluister deze afleveringIn this podcast, John DesJardins, field CTO and VP solution architecture at Hazelcast, sat down with InfoQ podcast co-host Daniel Bryant. Topics discussed included: how in-memory data grids have evolved, use cases at the edge (IoT, ML inference), integration of stream processing APIs and techniques, and how data grids can be used within application modernization. Lees meer
Listen to the podcast for more.
Curated transcript and more information on the https://bit.ly/2FtBIgp
Follow us on Facebook, Twitter, LinkedIn, Youtube: @InfoQ
Follow us on Instagram: @infoqdotcomStay informed on emerging trends, peer-validated early adoption of technologies, and architectural best practices. Subscribe to The Software Architects’ Newsletter: www.infoq.com/software-architects-newsletter/
-
Akhilesh Gupta on the Architecture of LinkedIn’s Real-time Messaging Platform
7 september 2020 · 22 min
Beluister deze afleveringCharles Humble talks to Akhilesh Gupta, the technical lead for LinkedIn's real-time delivery infrastructure, and also LinkedIn messaging. They discuss the architecture behind LinkedIn’s real-time platform, its building blocks, the frameworks used and other technical details. Lees meer
Listen to the podcast for more.
Curated transcript and more information on the https://bit.ly/2DujU3U
Follow us on Facebook, Twitter, LinkedIn, Youtube: @InfoQ
Follow us on Instagram: @infoqdotcomStay informed on emerging trends, peer-validated early adoption of technologies, and architectural best practices. Subscribe to The Software Architects’ Newsletter: www.infoq.com/software-architects-newsletter/
-
Yan Cui on Serverless Orchestration & Choreography, Distributed Tracking, Cold Starts, and more
1 september 2020 · 24 min
Beluister deze afleveringToday on the InfoQ Podcast, Yan Cui (a long time AWS Lambda user and consultant) and Wes Reisz discuss serverless architectures. The conversation starts by focusing on architectural patterns around choreography and orchestration. From there, the two move into updates on the current state of serverless cold start times, distributed tracing, and state. Today’s podcast, while not specific to AWS, does lean heavily on Yan’s expertise with AWS and AWS Lambda. Lees meer
Listen to the podcast for more.
Curated transcript and more information on the https://bit.ly/2YPOLj5
Follow us on Facebook, Twitter, LinkedIn, Youtube: @InfoQ
Follow us on Instagram: @infoqdotcomStay informed on emerging trends, peer-validated early adoption of technologies, and architectural best practices. Subscribe to The Software Architects’ Newsletter: www.infoq.com/software-architects-newsletter/
-
Liran Haimovitch on Understandability, Complexity, and Live Debugging
21 augustus 2020 · 23 min
Beluister deze afleveringIn this podcast, Liran Haimovitch, CTO at Rookout, sat down with InfoQ podcast co-host Daniel Bryant. Topics discussed included: the concept of “understandability” and how this relates to building modern software systems, how complexity impacts a system’s understandability, and the benefits of live debugging tooling. Lees meer
Listen to the podcast for more.
Curated transcript and more information on the podcast: https://bit.ly/3ggFE0r
Follow us on Facebook, Twitter, LinkedIn, Youtube: @InfoQ
Follow us on Instagram: @infoqdotcomStay informed on emerging trends, peer-validated early adoption of technologies, and architectural best practices. Subscribe to The Software Architects’ Newsletter: https://www.infoq.com/software-architects-newsletter/
-
Ana Medina on Chaos Engineering, Game Days, and Learning
10 augustus 2020 · 30 min
Beluister deze afleveringIn this podcast, Ana Medina, senior chaos engineer at Gremlin, sat down with InfoQ podcast co-host Daniel Bryant. Topics discussed included: how enterprise organisations are adopting chaos engineering with the requirements for guardrails and the need for “status checks” to ensure pre-experiment system health; how to run game days or IT fire drills when everyone is working remotely; and why teams should continually invest in learning from past incidents and preparing for inevitable failures within systems. Lees meer
Listen to the podcast for more.
Curated transcript and more information on the podcast: https://bit.ly/2Pmsq7F
Follow us on Facebook, Twitter, LinkedIn, Youtube: @InfoQ
Follow us on Instagram: @infoqdotcomStay informed on emerging trends, peer-validated early adoption of technologies, and architectural best practices. Subscribe to The Software Architects’ Newsletter: https://www.infoq.com/software-architects-newsletter/
-
Stefan Prodan on Progressive Delivery, Flagger, and GitOps
28 juli 2020 · 38 min
Beluister deze afleveringIn this podcast, Stefan Prodan, developer experience engineer at Weaveworks and creator of the Flagger project, sat down with InfoQ podcast co-host Daniel Bryant. Topics discussed included: how progressive delivery extends the core ideas of continuous delivery; how the open source Flagger Kubernetes operator can be used to implement a progressive delivery strategy via canary releasing with an API gateway or service mesh; and the new “GitOps toolkit” that has evolved from the Flux continuous delivery operator. Lees meer
Listen to the podcast for more.
Curated transcript and more information on the podcast: https://bit.ly/3f9CZ8p
Follow us on Facebook, Twitter, LinkedIn, Youtube: @InfoQ
Follow us on Instagram: @infoqdotcomStay informed on emerging trends, peer-validated early adoption of technologies, and architectural best practices. Subscribe to The Software Architects’ Newsletter: https://www.infoq.com/software-architects-newsletter/
-
Rancher on Hybrid Cloud, Kubernetes at the Edge, and Open Standards
10 juli 2020 · 32 min
Beluister deze afleveringIn this podcast, Shannon Williams, co-founder and president at Rancher Labs and Darren Shepherd, co-founder and CTO at Rancher Labs sat down with InfoQ podcast co-host Daniel Bryant. Topics discussed included: the adoption of hybrid cloud across organisations, the evolution of Kubernetes as a key abstraction for portability and cross-cloud security, running thousands of Kubernetes clusters at the edge, and the value of open standards. Lees meer
Why listen to this podcast
- Organisations are adopting hybrid cloud strategies. The use of containers to package and run applications across clouds has seen large adoption over the past five years. Containers and Kubernetes are everywhere: the datacenter, the edge, embedded systems, and other locations.
- Two enterprise use cases for Kubernetes stand out: providing standardised abstractions and APIs to increase portability across vendors and cloud platforms; and providing a framework and homogenised foundation on which to build and implement (cross cloud) security solutions.
- Open standards support interoperability and drive innovation. The CNCF is becoming the natural home for open cloud technologies. The Rancher team have donated Longhorn, their cloud-native distributed storage platform for Kubernetes that was recently announced as generally available, to the CNCF.
- With the success of lightweight Kubernetes distributions, such as Rancher’s K3s, engineers are starting to deploy standalone Kubernetes clusters “by the thousands” to edge locations. Rancher has recently released Fleet, a new open source project that is focused on managing large collections (“fleets”) of Kubernetes clusters.
- Many developers and end users of Kubernetes simply want a platform-as-a-service (PaaS)-like experience. The next 12 months will see the community focus on the simplification of the Kubernetes ecosystem.More on this: Quick scan our curated show notes on InfoQ https://bit.ly/31ZjYT1
You can also subscribe to the InfoQ newsletter to receive wee… -
Nora Jones on Resilience Engineering, Mental Models, and Learning from Incidents
3 juli 2020 · 36 min
Beluister deze afleveringIn this podcast, Nora Jones, Co-Founder and CEO at Jeli and co-author of O’Reilly’s “Chaos Engineering: System Resiliency in Practice”, sat down with InfoQ podcast co-host Daniel Bryant. Topics discussed included: chaos engineering and resilience engineering, planning and running effective chaos experiments, and learning from incidents. Lees meer
Why listen to this podcast:
- The chaos engineering and resilience engineering fields, although inextricably linked, are often incorrectly conflated. Resilience engineering is focused on “identifying and then enhancing the positive capabilities of people in organizations that allow them to adapt effectively and safely under varying circumstances.”
- The UX of internal or engineering-focused tooling, such as chaos experimentation tooling, is extremely important. However, engineers that create these tools often overlook the value of UX, or don’t have the relevant skills in user design research to undertake this.
- We all work in socio-technical systems. It is important to take the time to understand both aspects. Developing empathy and working alongside teams that you are trying to influence is essential. It is extremely important to continually work to build correct “mental models” of a system.
- The before and after of running a chaos experiment is as important as running the experiment itself. However, the aspects of planning, creating effective hypotheses, and analysing and disseminating the results are often under-resourced.
- Incident analysis can be a catalyst to help you understand more about your system. The Learning from Incidents website, alongside books such as Sidney Dekker’s The Field Guide to Understanding Human Error and Scott Snook’s Friendly Fire, can provide excellent background information to these topics.More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2C4R6xL
You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software develop… -
Rob Skillington on Metrics Collection, Uber’s M3, and OpenMetrics
26 juni 2020 · 33 min
Beluister deze afleveringIn this podcast, Rob Skillington, co-founder and CTO at Chronosphere, sat down with InfoQ podcast co-host Daniel Bryant. Topics discussed included: metrics collection at scale, multi-dimensional metrics and high-cardinality, developer experience with platform tooling, and open standards related to observability. Lees meer
Why listen to this podcast:
- Over the past ten years the requirements related to monitoring and alerting, and the approach taken to implement this, has changed considerably. Compute is now ephemeral and dynamic, services are more numerous, and engineers want to instrument more things. Scalability of a monitoring solution is vitally important.
- One of the challenges with metric data is the limited information for providing context for collected values. This can be solved by using multi-dimensional metrics. Dimensions of a metric are name-value pairs that carry additional data to describe the metric value. High dimensionality can lead to high cardinality.
- Uber’s M3 metrics collection system initially used open source components such as Cassandra and ElasticSearch for storage and indexing. As the scale of usage of M3 increased, these OSS components were gradually replaced by custom components, such as M3DB.
- Building an effective user experience for operational tooling, especially observability-foused tooling, is vitally important. Engineers will be interacting with these tools on a daily basis. They will also be relying on these tools for both alerting and being able to locate and understand what is occurring during production issues.
- Open standards are vitally important for interoperability. The OpenMetrics project is an effort to create an open standard for transmitting metrics at scale, with support for both text representation and protocol buffers.More on this: Quick scan our curated show notes on InfoQ https://bit.ly/3i3dMPO
You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional softwar… -
Johnny Boursiquot on Serverless Go and Site Reliability Engineering at Heroku
19 juni 2020 · 41 min
Beluister deze afleveringIn this podcast, Johnny Boursiquot, Site Reliability Engineer at Heroku, sat down with InfoQ podcast co-host Daniel Bryant and discussed topics that included: why Go is a useful language for building Function-as-a-Service (FaaS) style applications; how Heroku implement the role of Site Reliability Engineer (SRE); and why the ability to teach is such a valuable skill. Lees meer
Why listen to this podcast:
- Go is a useful language for building Function-as-a-Service (FaaS) style applications. The ability to build Go applications into a static binary reduces the need for dependency management, and the quick runtime and application start time is good for initiation and scaling
- The FaaS development toolchain has improved over the years. Many cloud providers now provide local runtimes, e.g. AWS SAM Local, and service simulators, e.g. LocalStack. Testing in production is facilitated by the ability to do dark launches and canary releasing at the ingress/API gateway
- Developing “serverless” applications typically does not remove the need for operational expertise on a development team. Designing systems appropriately and getting the most out of the runtime (with minimal cost) requires knowledge of the underlying infrastructure components
- The role of Site Reliability Engineering (SRE) looks different across practically every organisation. The Heroku SRE team have adapted well-established patterns and practices into their roles. They act as “diplomats”, working closely with product teams to share knowledge around operational best practices
- The ability to teach is a valuable skill, regardless of your job. Teaching people to code or to embrace important operational principles is extremely rewarding. - Engineers who teach must seek to escape the pull of their ego; by focusing on the needs of the people you are teaching, much more progress can be made.More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2UV0tqK
You can also subscribe to the InfoQ newsletter to… -
Matt Debergalis on GraphQL and Data Modelling in the Enterprise
13 juni 2020 · 34 min
Beluister deze afleveringIn this podcast, Matt Debergalis, Founder and CTO at Apollo, sat down with InfoQ podcast co-host Daniel Bryant. Topics discussed included: the motivations for GraphQL, the Apollo Data Graph platform, data modelling in an enterprise context, and how incrementally adopting GraphQL can help with decoupling the evolution of frontend and backend systems. Lees meer
Why listen to this podcast:
- The challenges of defining client-side-friendly data models, building maintainable and composable backend APIs, and moving data from the cloud to a client application contributes to making modern software development difficult and time consuming.
- GraphQL is an open-source data query and manipulation language for APIs, and a runtime for fulfilling queries with existing data. GraphQL lets application developers describe the data they need and bring that data into the screens that they are building for their users.
- The Apollo Data Graph platform is a middleware layer that provides a way of decoupling the core business APIs from the client-side consumption patterns. Apollo can implement cross-cutting concerns, such as transaction management, which mitigates the need to implement this in (potentially multiple) client-side applications.
- Apollo makes it possible to build a “data graph”: a series of graphs that are composed from an organisation’s data for use within client-side applications. A data graph is especially valuable in larger enterprises because it is here that many (money making) systems with existing APIs need to be combined to meet new business requirements.
- GraphQL can be adopted in an incremental fashion. To begin adoption just build the simplest possible graph that matches the needs of the first application, the first screen, or the first component that is required to transition over to the graph. Then let that graph evolve.More on this: Quick scan our curated show notes on InfoQ https://bit.ly/3fhaeqP
You can also subscribe to the InfoQ newsletter to receive weekly upd… -
Lin Sun and Neeraj Poddar on Istio, Wasm, and the Future of Service Mesh
31 mei 2020 · 37 min
Beluister deze afleveringIn this podcast, Lin Sun, senior technical staff member and master inventor at IBM, and Neeraj Poddar, engineering lead and architect at Aspen Mesh, sat down with InfoQ co-host Daniel Bryant. Topics discussed included: the evolution of service mesh data planes and control planes, the new Istio 1.5 architectures, Istio WebAssembly extension support, and the future of service mesh technology. Lees meer
Why listen to this podcast:
- A service mesh in one implementation approach to provide service discovery, traffic management, and cross-cutting communication concerns that engineers see when they adopt (micro)service-based.
- The data plane of most modern service mesh implementations run out-of-process as a proxy sidecar. This has evolved from library based implementations, such as Airbnb’s SmartStack or Netflix’s OSS libraries.
- The recent release of Istio 1.5 saw the deployment packaging of the control plane move from a microservice-based approach to that of a monolithic implementation, named “istiod”.
- Istio now also supports data plane extensions written in WebAssembly (Wasm). These extensions can modify requests and responses and perform out-of-band actions, such as authentication and authorization.
- Standardisations like the Service Mesh Interface (SMI) can add a lot of value, but the user requirements, common use cases, and the core abstractions of the underlying technology must be well understood.
- Multi-cluster and mesh expansion (out-of-cluster) support is continually improving in Istio and many other service mesh implementations. -
Sam Newman: Monolith to Microservices
25 mei 2020 · 44 min
Beluister deze afleveringToday on the InfoQ Podcast, Wes Reisz talks with one of the thought leaders in Microservices, CI/CD, and Cloud -- Sam Newman. The podcast covers many of the topics, techniques, and patterns that Sam writes about in his latest book, Monolith to Microservices: Evolutionary Patterns to Transform Your Monolith. Topics covered in the podcast include understanding the problem you’re trying to solve, organizational/people changes when it comes to microservice architectures, database strategies for decomposing monolithic datastores, and why we’re seeing projects reverting from microservices to monoliths. Lees meer
Why listen to this podcast:
- Fundamentally, microservices are distributed systems. Distributed systems have baggage (complexity) that comes along with them. The best way to deal with this complexity is not to address it. Try to solve the problem in other ways before choosing to take an organization to microservices.
- A common issue that large enterprises run into that might be a strong indicator for implementing microservices occurs when lots of developers are working on a given problem and they’re getting in each other’s way.
- A useful structure to follow with microservices is to make sure each service is owned by exactly one team. One team can own more than one service but having clear ownership of who owns a service helps in some of the operational challenges with microservices.
- A release train should be a stop in the journey towards continuous delivery. It’s not the destination. If you find that you can only release in a release train, you are likely building a distributed monolith.
- There are challenges of operating microservices when the end customer has to operate and manage it. These challenges are part of why we’re seeing projects move from microservices to process monoliths.More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2XvGzmF
You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from pr… -
Tracy Miranda on the Continuous Delivery Foundation, Interoperability, and Open Standards
15 mei 2020 · 24 min
Beluister deze afleveringIn this podcast, Tracy Miranda sat down with InfoQ podcast co-host Daniel Bryant. Miranda, Director of Open Source Community at CloudBees, and board chair at the Continuous Delivery Foundation (CDF), discussed topics that included: the aims of the CDF and an outline of the current hosted projects, the need for open standards and interoperability in the CD space, and the benefits offered by progressive delivery and software supply chain management. Lees meer
Why listen to this podcast:
- The Continuous Delivery Foundation (CDF) serves as the vendor-neutral home of many projects within continuous delivery space, including: Jenkins, Jenkins X, Spinnaker, Tekton, and Screwdriver.cd
- Jenkins X is a Kubernetes-native continuous delivery solution for cloud applications. This project uses a completely new architecture and code base in comparison with the original Jenkins project.
- Spinnaker is an open source, multi-cloud continuous delivery platform. The Tekton Pipelines project provides Kubernetes-style custom resources for declaring continuous integration and delivery pipelines. Spinnaker can use Tekton as its pipeline engine.
- In addition to providing a neutral home for projects within the CD space, the CDF is also aiming to help define appropriate terminology, open standards, and abstractions. This will assist with interoperability between CD components, and also promote innovation in the areas that can provide the most value.
- The CDF is also aiming to facilitate software testing, progressive delivery, and software supply chain management. Wide ranging topics such as observability and security are important will play an important role here.More on this: Quick scan our curated show notes on InfoQ https://bit.ly/3bzoYiz
You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVqSubscribe: www.youtube.com/infoq
Like InfoQ on Facebook: bit.ly/2jmlyG8
Follow on Twitter: twitter.com/In… -
Marty Abbott and Tanya Cordrey on Microservices, Availability, and Managing Risk
8 mei 2020 · 37 min
Beluister deze afleveringIn this podcast, Marty Abbott and Tanya Cordrey sat down with InfoQ podcast co-host Daniel Bryant. Abbott, CEO and co-founder of AKF Partners, and Cordrey, partner at AKF Partners, discussed topics that included: their learning from working together in the early days of eBay, why and how to avoid creating software systems that are composed of deep call chains of microservices, and how to build effective product teams. Lees meer
Why listen to this podcast:
- First introduced in the book “The Art of Scalability”, the AKF Scale Cube is a model for segmenting software components, defining microservices, and scaling products. It also creates a common language for teams to discuss scale related options in designing solutions.
- The microservice architectural pattern is best used for implementing the “breadth” of business functionality. Engineers should avoid building deep call chains of services, as this can increase the probability of failure, and can also increase the challenges of locating and diagnosing issues. Code libraries can often be used more effectively to implement “depth” within services.
- The AKF Availability Cube is a new model to guide discussions on how to achieve high availability. This model can also be used as a mathematical tool to evaluate the theoretical “as designed” availability of existing systems.
- Building products using cross-functional teams is an effective approach. However, care should be taken not to accidentally create unnecessarily large teams, as this can add communication and coordination friction to the delivery process.
- Teams should make a conscious choice to adopt new technologies, and understand the benefits and tradeoffs with doing so. Managing risk, and in particular, technology lifespan risk, is an important part of the value engineers provide to the business.More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2AbpZjT
You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topic… -
Dave Sudia on Migrating From a PaaS to a Kubernetes-Based Platform
17 april 2020 · 31 min
Beluister deze afleveringIn this podcast, Daniel Bryant sat down with Dave Sudia, senior DevOps engineer at GoSpotCheck. Topics discussed included: the benefits of PaaS; building a platform with Kubernetes as the foundation; selecting open source components and open standards in order to facilitate the evolution of a platform; and why care should be taken to prioritize the developer experience and create self-service operation of the platform. Lees meer
Why listen to this podcast:
- When starting a business and searching for product-market fit, creating an application using a monolithic code base deployed onto a commercial Platform as a Service (PaaS) product is a very effective way of iterating fast and minimising operational costs.
- There may come a point where the PaaS cannot provide bespoke requirements, or it has trouble scaling, or the costs become prohibitive. At this point many teams choose to build a custom platform using cloud technologies, such as Kubernetes.
- Building a Kubernetes platform can be an effective solution, but appropriate effort needs to be put into designing, building, and maintaining the platform. The platform effectively becomes another product within the business that must be managed accordingly.
- Embracing open standards provides many benefits, especially for the long term. Implementations that are consumed through well-defined interfaces and abstraction can be more readily swapped at a later point in time. It is also generally easier to integrate components that share common interfaces.
- Attention and resources must be provided to create an effective developer experience for the platform. It is essential to prioritize self-service operations, and also to understand the core requirements of the engineers and QA specialists that will be using the platform during their daily work.
- Establishing an effective continuous delivery pipeline can enable more repeatable and scalable testing of applications, and also allows the codification of cross-functional requirements.… -
Peter Bourgon on CRDTs and State at the Edge
3 april 2020 · 37 min
Beluister deze afleveringToday on The InfoQ Podcast, Wes Reisz talks with Peter Bourgon. Peter is a distributed system engineer working on Fastly. His area of interest is around coordination free replicated systems. The two engineers talk about the space of Conflict-Free Replicated Data Types (CRDTs) specifically in the context of edge compute. Topics covered on the podcast include Edge compute, CRDTs, CAP Theorem, and challenges around building distributed systems. Lees meer
Why listen to this podcast:
- CRDTs (conflict-free replicated data types) are a class of coordination free replication systems (or systems that don’t require strategies such as leader election).
- An easy way to think of a CRDT is as an associative, commutative, and idempotent data structure plus the operations needed to use it.
The edge is an overloaded term that people tend to define based on where they sit on a spectrum between the customer and the data center. Fastly’s edge is away from the data center and but not to the telephone pole or handset.
- RAFT and Gossip are two alternative approaches to using a coordination free replication system like CRDTs.
To get the properties of a CRDT and have useful data types, you have to pay a cost in size and often bytes on the wire. These are challenges that continue to need solutions.
- Modern Distributed systems and data structures like CRDTs require you to start thinking about state in the system itself. It’s not unusual for a system today to give you multiple results back that the system will have to handle or merge.More on this: Quick scan our curated show notes on InfoQ https://bit.ly/3dTZOO4
You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVqSubscribe: www.youtube.com/infoq
Like InfoQ on Facebook: bit.ly/2jmlyG8
Follow on Twitter: twitter.com/InfoQ
Follow on LinkedIn: www.linkedin.com/company/infoq
Check the landing page on InfoQ: https://bit.ly/3dTZOO4 -
Joe Duffy on Infrastructure as Code, Pulumi, and Multi-Cloud
27 maart 2020 · 28 min
Beluister deze afleveringIn this podcast, Daniel Bryant sat down with Joe Duffy, founder and CEO at Pulumi, and discussed several infrastructure-themed topics: the evolution of infrastructure as code (IaC), the way in which the open source Pulumi framework allows engineers to write IaC using general purpose programming languages such as JavaScript and Go, and the future of multi-cloud environments. Lees meer
Why listen to this podcast:
● Infrastructure as Code (IaC) enables engineers to programmatically define the configuration and provisioning of computing infrastructure, on-premises hardware, and cloud services.
● Traditional IaC tools were often imperative, requiring engineers to define and enumerate the necessary steps and SDK calls in order to configure the underlying infrastructure.
● Modern IaC tools like HashiCorp’s Terraform, AWS CloudFormation and other related cloud vendor tooling enable engineers to write declarative code to define a required state of the infrastructure. The tools parse the declarative configuration and take appropriate action to enact the specified state, for example, calling SDKs and APIs, verifying results, iterating etc.
● Pulumi is an open source framework that enables engineers to define IaC using general purpose programming languages, such as Node, Python, .NET Core, and Go.
● Pulumi allows imperative specification of IaC. Engineers can use their favourite language-specific features, idioms, and patterns. The use of language modules, packages, and libraries can also enable code reuse.
● Under the hood, Pulumi transforms code written in the supported languages to a declarative specification model. This model is then used to enact the required infrastructure state.
● Frameworks like Pulumi enable engineers to deploy and configure infrastructure across multiple cloud vendors and services (including Kubernetes clusters).More on this: Quick scan our curated show notes on InfoQ https://bit.ly/3dyxVee
You can also subscribe to the InfoQ newsletter to receive weekly up… -
Dylan Schiemann on the Evolution of Dojo, Web Components and Trends in the Web Development Landscape
20 maart 2020 · 33 min
Beluister deze afleveringIn this podcast Charles Humble spoke to Dylan Schiemann, co-creator of Dojo and InfoQ’s JavaScript and Web Development lead editor, about the history and current state of Dojo, and key emerging trends in the JavaScript landscape today. Key topics include Dojo’s adoption of Typescript, web components, and client-side libraries such as Svelte and Stencil. Lees meer
Why listen to this podcast:
- Modern Dojo (2.0 and upwards) is focussed on being a very small, opinionated reactive framework, but with a lot of the components you need to build a modern JavaScript application built in.
- The framework tries to align closely to standards, for example using Web Components extensively for UI components, alongside ES modules and promises. The use of standards, as well as the convergence towards the reactive programming model for web UI, has improved interoperability, though there are some limitations such as the lack of an easy way to share resources across web components.
- Dojo was one of the first frameworks to make the decision to switch to Typescript, though it took some time to make that transition. The switch was mainly motivated by TypeScript’s support for interfaces, but it wasn’t until Typescript 2.6 they felt able to ship Dojo 2.
- On the client side we’re paying close attention to Svelte and Stencil as two particularly interesting client-side frameworks.
- We’ve moved Web Components from early adopter to early majority on the trend report, based on the fact that all browsers accept IE now natively support it, but also large companies such as Apple, Nike and ESPN are deploying web components and their sites. Apple’s iTunes implementation, for example, now uses web components.More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2Qy75Jr
You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVqSubscribe: www.youtube.com/infoq
Like InfoQ on Facebook: bit.ly/2jmlyG… -
Gareth Rushgrove on Kubernetes as a Platform, Applications, and Security
13 maart 2020 · 31 min
Beluister deze afleveringIn this podcast, Daniel Bryant sat down with Gareth Rushgrove, Director of Product Management at Snyk. Topics covered included Kubernetes as a platform, application abstractions, continuous delivery, and implementing good security practices in the cloud native space. Lees meer
Why listen to this podcast:
- The value provided by Kubernetes depends on an organisation’s context. Kubernetes acts as both a series of lower-level building blocks for a platform, and also as a very powerful API for deploying and operating container-based applications.
- Kubernetes provides several useful abstractions for engineers. For example, Pods, Deployments, and Services. However, Kubernetes doesn’t have an “application”-focused abstraction. Tools such as Helm and specifications like the Cloud Native Application Bundle (CNAB) are driving innovation in this space.
- There is a large amount of open source Kubernetes tooling. This has been created by a range of vendors, groups, and individuals. Encouraging this diverse mix of participation is beneficial for the long-term health of the ecosystem.
- The Cloud Native Computing Foundation (CNCF) provides a space for people to collaborate regardless of their current organisational affiliations.
- Defining appropriate standards within the cloud native space is useful for enabling interoperability and providing common foundations for others to innovate on top of.
- Security challenges within IT are socio-technical. Security teams working with cloud native technologies will benefit from continual learning, developing new skills, and researching new tools. For example, the defaults of Kubernetes aren’t necessarily secure, but this can be readily addressed with appropriate configuration.More on this: Quick scan our curated show notes on InfoQ https://bit.ly/38PLPFb
You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVqSubscribe: www.youtube.com/infoq
Like Inf… -
Luca Mezzalira on Micro Frontends at DAZN
9 maart 2020 · 35 min
Beluister deze aflevering- A Micro frontends is an approach to developing frontends that attempts to take some of the same benefits from Microservices and apply them to frontend development.
- Microfront ends can be developed with different technologies and ownership of components on a single view. However, DAZN took a vertical approach to build them. Each Micro frontend is loaded into an app shell that offers an API for crosscutting concerns. Only one Micro frontend is loaded at a time into the app shell.
- The ‘Inverse Conway Maneuver’ recommends evolving your team and organizational structure to create the architecture you want.
- DAZN derisks deployments by using canaries implemented with Lambda at the Edge on Cloudfront. For code deployments, each of the Micro frontends can be introduced with a limited scope and then expanded once proven stable. Lees meerMore on this: Quick scan our curated show notes on InfoQ https://bit.ly/38BQAC0
You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVqSubscribe: www.youtube.com/infoq
Like InfoQ on Facebook: bit.ly/2jmlyG8
Follow on Twitter: twitter.com/InfoQ
Follow on LinkedIn: www.linkedin.com/company/infoq
Check the landing page on InfoQ: https://bit.ly/38BQAC0 -
Zhamak Dehghani on Data Mesh, Domain-Oriented Data, and Building Data Platforms
2 maart 2020 · 34 min
Beluister deze afleveringIn this podcast, Daniel Bryant sat down with Zhamak Dehghani, principal consultant, member of technical advisory board, and portfolio director at ThoughtWorks. Topics discussed included: the motivations for becoming a data-driven organization; the challenges of adapting legacy data platforms and ETL jobs; and how to design and build the next generation of data platforms using ideas from domain-driven design and product thinking, and modern platform principles such as self-service workflows. Lees meer
Why listen to this podcast:
- Becoming a data-driven organization remains one of the top strategic goals of many organizations. Being able to rapidly run experiments and efficiently analyse the resulting data can provide a competitive advantage.
- There are several “architecture failure modes” within existing enterprise data platforms. They are centralized and monolithic. The composition of data pipelines are often highly-coupled, meaning that a change to the data format will require a cascade of changes throughout the pipeline. And finally, the ownership of data platforms is often siloed and hyper-specialized.
- The next generation of enterprise data platform architecture requires a paradigm shift towards ubiquitous data with a distributed data mesh.
Instead of flowing the data from domains into a centrally owned data lake or platform, domains need to host and serve their domain datasets in an easily consumable way.
- Domain data teams must apply product thinking to the datasets that they provide; considering their data assets as their products, and the rest of the organization's data scientists, ML and data engineers as their customers.
The key to building the data infrastructure as a platform is (a) to not include any domain specific concepts or business logic, keeping it domain agnostic, and (b) make sure the platform hides all the underlying complexity and provides the data infrastructure components in a self-service manner.More on this: Quick scan our curated show note…
-
Brittany Postnikoff on Security, Privacy, and Social Engineering with Robots
21 februari 2020 · 23 min
Beluister deze afleveringIn this podcast, Daniel Bryant sat down with Brittany Postnikoff, a computer systems analyst specialising on the topics of robotics, embedded systems, and human-robot interaction. Topics discussed included: the rise of robotics and human-robot interaction within modern life, the security and privacy risks of robots used within this context, and the potential for robots to be used to socially engineer people. Lees meer
Why listen to this podcast:
- Physical robots are becoming increasingly common in everyday life, for example, offering directions in airports, cleaning the floor in peoples’ homes, and acting as toys for children.
- People often imbue these robots with human qualities, and they trust the authority granted to a robot.
- Social engineering can involve the psychological manipulation of people into performing actions or divulging confidential information. This can be stereotyped by the traditional “con”.
- As people are interacting with robots in a more human-like way, this can mean that robots can be used for social engineering.
- A key takeaway for creators of robots and the associated software is the need to develop a deeper awareness of security and privacy issues.
- Software included within robots should be patched to the latest version, and any data that is being stored or transmitted should be encrypted.
- Creators should also take care when thinking about the human-robot UX, and explore the potential for unintended consequences if the robot is co-opted into doing bad things.More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2v5QTav
You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVqSubscribe: www.youtube.com/infoq
Like InfoQ on Facebook: bit.ly/2jmlyG8
Follow on Twitter: twitter.com/InfoQ
Follow on LinkedIn: www.linkedin.com/company/infoq
Check the landing page on InfoQ: https://bit.ly/2v5QTav -
Anurag Goel on Cloud Native Platforms, Developer Experience, and Scaling Kubernetes
7 februari 2020 · 27 min
Beluister deze afleveringIn this podcast, Daniel Bryant sat down with Anurag Goel, Founder and CEO of Render. Topics covered included: the evolution of cloud platforms; simplifying developer experience; running large-scale workloads on Kubernetes; and the future of tooling and platforms within the cloud native computing space. Lees meer
Why listen to this podcast:
- Render is aiming to be the next generation of cloud provider. Developers deploy and manage applications via a Platform-as-a-Service (PaaS) -like experience using custom simplified YAML configuration.
- Render is built on top of Kubernetes, but the internals and configuration of this orchestration framework is not exposed to end user developers.
- Many large scale usages of traditional cloud vendor platforms require the formation of specialised in-house “DevOps” teams. The provision of virtualisation and API-driven operation via the cloud providers was revolutionary, but it didn’t fundamentally change the existing platform paradigm.
- Arguably platform usability may have taken a step back with the arrival of public cloud vendor platforms. For example, developers may just want to write code, and not have to write complicated deployment descriptors. Operations team may want to focus on supporting engineers and advising on performance and scale, rather maintaining cloud provisioning scripts.
- The Render team are planning to run all future workloads of self-managed Kubernetes, rather than use a hosted offering, due to them experiencing implementation bugs when running their clusters at medium-to-large scale.
- The Cloud Native Computing Foundation (CNCF) is encouraging large amounts of innovation within the cloud platform space. However, due to the Cambrian explosion of the Cloud Native Landscape over the past several years, there must surely be consolidation of tools, platforms, and vendors in the near future.More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2H2lY1s
You can also subscribe to the InfoQ newsletter t…
-
Greg Law on Debugging, Record & Replay of Data, and Hyper-Observability
31 januari 2020 · 31 min
Beluister deze afleveringIn this podcast, Daniel Bryant sat down with Greg Law, CTO at Undo. Topics discussed included: the challenges with debugging modern software systems, the need for “hyper-observability” and the benefit of being able to record and replay exact application execution; and the challenges with implementing the capture of nondeterministic system data in Undo’s LiveRecorder product for JVM-based languages that are Just-In-Time (JIT) compiled. Lees meer
Why listen to this podcast:
- Understanding modern software systems can be very challenging, especially when the system is not doing what is expected. When debugging an issue, being able to observe a system and look at logging output is valuable, but it doesn’t always provide all of the information a developer needs. Instead we may need “hyper observability”; the ability to “zoom into” bugs and replay an exact execution.
- Being able to record all nondeterministic stimuli to an application -- such as user input, network traffic, interprocess signals, and threading operations -- allows for the replay of an exact execution of an application for debugging purposes. Execution can be paused, rewound, and replayed, and additional logging data can be added ad hoc.
- Undo’s LiveRecorder allows for the capture of this nondeterministic data, and this can be exported and shared among development teams. The UndoDB debugger, which is based on the GNU Project Debugger, supports the loading of this data and the execution and debugging in forwards and reverse execution of the application. There is also support for other debuggers, such as that included within IntelliJ IDEA.
- Advanced techniques like multi-process correlation reveal the order in which processes and threads alter data structures in shared memory, and thread fuzzing randomizes thread execution to reveal race conditions and other multi-threading defects.
- The challenges of using this type of technology when debugging (micro)service-based application lies within the user experience i.… -
Idit Levine Discussing Gloo, Service Mesh Interface, and Web Assembly Hub
24 januari 2020 · 39 min
Beluister deze afleveringToday on The InfoQ Podcast, Wes Reisz speaks with CEO and founder of Solo Idit Levine. The two discuss the Three Pillars of Solo around Gloo, their API gateway, interoperability of service meshes (including the work on Service Mesh Interface), and on extending Envoy with Web Assembly (and the recently announced Web Assembly Hub). Lees meer
Why listen to this podcast:
- Gloo is a Kubernetes-native ingress controller and API gateway. It’s built on top of Envoy and at its core is open source.
- The Service Mesh Interface (SMI) is a specification for service meshes that runs on Kubernetes. It defines a common standard that can be implemented by a variety of providers. The idea of SMI is it’s an abstraction on top of service meshes, so that you can use one language to configure them all.
- Autopilot is an open-source Kubernetes operator that allows developers to extend a service mesh control plane.
- Lua has been commonly used to extend the service mesh data plane. Led by Google and the Envoy community, web assembly is becoming the preferred way of extending the data plane. Web assembly allows you to write Envoy extensions in any language while still being sandboxed and performant.
- WebAssembly Hub is a service for building, deploying, sharing, and discovering Wasm extensions for Envoy.
- Wasme is a docker like an open-source commandline tool from Solo to simplify the building, pushing, pulling, and deploying Envoy Web Assembly Filters.More on this: Quick scan our curated show notes on InfoQ https://bit.ly/37sYIoE
You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVqSubscribe: www.youtube.com/infoq
Like InfoQ on Facebook: bit.ly/2jmlyG8
Follow on Twitter: twitter.com/InfoQ
Follow on LinkedIn: www.linkedin.com/company/infoq
Check the landing page on InfoQ: https://bit.ly/37sYIoE -
Gunnar Morling on Change Data Capture and Debezium
17 januari 2020 · 29 min
Beluister deze afleveringToday, on The InfoQ Podcast, Wes Reisz talks with Gunnar Morling. Gunnar is a software engineer at RedHat and leads the Debezium project. Debezium is an open-source distributed platform for change data capture (CDC). On the show, the two discuss the project and many of its use cases. Additionally, topics covered on the podcast include bootstrapping, configuration, challenges, debugging, and operational modes. The show wraps with long term strategic goals for the project. Lees meer
Why listen to this podcast:
- CDC is a set of software design patterns used to react to changing data in a data store. Used for things like internal changelogs, integrations, replication, and event streaming, CDC can be implemented leveraging queries or against the DB transaction log. Debezium leverages the transaction log to implement CDC and is extremely performant.
- Debezium has mature source and sink connectors for MySQL, SQL Server, and MongoDB. In addition, there are Incumbating connectors for Cassandra, Oracle, and DB2. Community sink connectors have been created for ElasticSearch.
- In a standard deployment, Debezium leverages a Kafka cluster by deploying connectors into Kafka Connect. The connectors establish a connection to the source database and then write changes to a Kafka topic.
- Debezium can be run in embedded mode. Embedded mode imports Java library into your own project and leverages callbacks for change events. The library approach allows Debezium implementations against other tools like AWS Kinesis or Azure's Event Hub. Going forward, there are plans to make a ready-made Debezium runtime.
- Out of the box, Debezium has a one-to-one mapping between tables and Kafka topic queues. The default approach exposes the internal table structure to the outside. One approach to address exposing DB internals is to leverage the Outbox Pattern. The Outbox Pattern uses a separate outbox table as a source. Inserts into your normal business logic tables also make writes to the outbox. Change … -
Kelsey Hightower on Extending Kubernetes, Event-Driven Architecture, and Learning
10 januari 2020 · 26 min
Beluister deze afleveringIn this podcast, Daniel Bryant sat down with Kelsey Hightower, Staff Developer Advocate at Google. Topics covered included: the extensibility of Kubernetes, and why it has become the platform that other platforms are being built on top of; creating event-driven architectures and deploying these onto Function-as-a-Service (FaaS) platforms like the Kubernetes-based Knative and Google Cloud Run; and the benefits of learning, sharing knowledge, and building communities. Lees meer
Why listen to this podcast:
- Kubernetes is a platform for building platforms. It may not be as opinionated as traditional Platform-as-a-Service (PaaS) offerings, but it has become popular due to its extensibility. There are PaaS-like solutions built on top of Kubernetes, such as OpenShift, Knative, and Cloud Run.
- The creation of common interfaces within Kubernetes -- such as Custom Resource Definitions (CRDs), Container Networking Interface (CNI), and Container Runtime Interface (CRI) -- enabled the adoption of the platform by vendors and the open source community without everyone needing to agree on exactly how to implement extensions.
- Although not every workload can be effectively implemented using an event-driven architecture, for those that can the Kubernetes-based Function-as-a-Service (FaaS) platforms like Knative and Cloud Run can handle a lot of the operational management tasks for developers.
- Engineers may be able to get ~90% of the “service mesh” traffic management functionality they need from using a simple proxy.
However, the separation of the control and data planes within modern service meshes, in combination with the rise in popularity of the sidecar deployment model, has provided many benefits within Kubernetes.
- A lot of learning within software development and information technology is transferable. If you spend time going deep in a technology when you begin your career, much of what you learn will be useful when you come to learn the next technology.More on this: Quick sca…
-
Katie Gamanji on Condé Nast’s Kubernetes Platform, Self-Service, and the Federation and Cluster APIs
3 januari 2020 · 31 min
Beluister deze afleveringIn this podcast, Daniel Bryant sat down with Katie Gamanji, Cloud Platform Engineer at Condé Nast International. Topics covered included: exploring the architecture of the Condé Nast Kubernetes-based platform; the importance of enabling self-service deployment for developers; and how the Kubernetes’ Federation API and Cluster API may enable more opportunities for platform automation. Lees meer
- Founded in the early 1900s, Condé Nast is a global media company that has recently migrated their application deployment platforms from individually-curated geographically-based platforms, to a standardised distributed platform based on Kubernetes and AWS.
- The Condé Nast engineering team create and manage their own Kubernetes clusters, currently using CoreOS’s/Red Hat’s Tectonic tool.
Self-service deployment of applications is managed via Helm Charts. - The platform team works closely with their “customer” developer teams in order to ensure their requirements are being met.
- The Kubernetes Federation API makes it easy to orchestrate the deployment of applications to multiple clusters. This works well for cookie-cutter style deployments that only require small configuration differences, such as scaling the number of running applications based on geographic traffic patterns.
- The Cluster API is a Kubernetes project to bring declarative APIs to cluster creation, configuration, and management. This enables more effective automation for cluster lifecycle management, and may provide more opportunities for multi-cloud Kubernetes use.
- The Condé Nast platform Kubernetes Ingress is handled by Traefik, due to the good Helm support and cloud integration (for example, AWS Route 53 and IAM rule synchronization). The platform team is exploring the use of service mesh for 2020.
- Abstractions, interfaces, and security will be interesting focal points for improvement in the Kubernetes ecosystem in 2020.More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2FeYPrE
You can al… -
Joseph Jacks on Commercial Open Source Software, RISC-V, and Disrupting the Application Layer
27 december 2019 · 26 min
Beluister deze afleveringIn this podcast, Daniel Bryant spoke to Joseph Jacks, Founder of OSS Capital and the Open Core Summit, and discussed topics including the open source and open core models, innovations within open source hardware and the RISC-V instruction set architecture, and current opportunities for disruption using commercial open source software. Lees meer
Why listen to this podcast:
- Recently, open source software and the open core business model have driven a lot of innovation and created a lot of value, particularly within the cloud “as-a-service” space.
- There has been some disagreement between the open source and commercially-focused communities, for example, in relation to the licencing models and how value is captured.
- The Open Core Summit (OCS) is a new conference focusing on the intersection of commercialisation and open source software that aims to facilitate discussion in this space.
- Organisations building around open source software can potentially look at large cloud vendors as partners. Public clouds can provide effective distribution, and typically focus on offering breadth of services rather than the depth of expertise that can be provided by a specialist company.
- RISC-V is an open-source hardware instruction set architecture (ISA) based on the well-established reduced instruction set computer (RISC) principles. Leveraging RISC-V can reduce the time and cost of customising chip designs.
- A lot of recent open source innovation has focused on the infrastructure layer within computing systems. This means that the application layer is now potentially ripe for disruption via commercial open source software.More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2rDfYYU
You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVqSubscribe: www.youtube.com/infoq
Like InfoQ on Facebook: bit.ly/2jmlyG8
Follow on Twitter: twitter.com/InfoQ
Follow on LinkedIn: www… -
The InfoQ Podcast Hosts Take a Look Back at 2019, Discussing Teal, Edge, Quantum Computing, and more
16 december 2019 · 33 min
Beluister deze afleveringIn this special year-end wrap-up podcast Wes Reisz, Shane Hastie, Daniel Bryant, and Charles Humble discuss what we’ve seen in 2019 and speculate a little on what we hope to see in 2020. Topics include business agility and Teal, what it means to be an ethical engineer, bringing your whole self to work, highlights from QCon and InfoQ during 2019, the rise of Python, and progress in quantum computing. Lees meer
Why listen to this podcast:
* Business agility is one of the major themes that the InfoQ team has seen emerge this year, with stronger emphasis on outcomes over outputs. We’ve also seen a growing interest in ethics and the ethical implications of the work we all do.
* On the programming languages front the rise of Python continues, driven largely by its popularity in data science.
* As Kubernetes cements its dominant position we’re hoping to see a simplification of the workflows associated with it, as well as in areas like observability.
* There have been several big announcements in quantum computing in the past year, and this is an area we continue to watch with interest.
* Another key trend for next year is edge computing. The edge of the cloud infrastructure has an amazing amount of available compute resource, as does the device edge.More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2Z0Q9OI
You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVqSubscribe: www.youtube.com/infoq
Like InfoQ on Facebook: bit.ly/2jmlyG8
Follow on Twitter: twitter.com/InfoQ
Follow on LinkedIn: www.linkedin.com/company/infoq
Check the landing page on InfoQ: https://bit.ly/2Z0Q9OI -
Josh Wills on Building Resilient Data Engineering and Machine Learning Products at Slack
9 december 2019 · 31 min
Beluister deze afleveringJosh Wills, a software engineer working on data engineering problems at Slack, discusses the Slack data architecture and how they build and observe their pipelines. Josh, along with color commentary such as the move from IC to manager (and back), discusses recommendations, tips, tools, and lessons Slack engineering teams discovered while building products like Slack Search. The podcast covers machine learning, observability, data engineering, and general practices for building highly resilient software. Lees meer
Why listen to this podcast:
- Slack has a philosophy of building only what they need. They have a don’t reinvent the wheel mindset.
- Slack was originally a PHP monolith. Today, it is largely Hack-lang, HHVM, and several Java and Go binarys. On the data side, application logs are in Thrift (there is a plan to migrate to protobuf). Events are processed through a Kafka cluster that handles 100,000s of events per second. Everything is kept in S3 with a large Hive metastore. EMR is spun up on demand. Presto, Airflow, Slack, Snowflake (business analytics), Quiver (key value store) are all used.
- ML worked best for Slack when it was used to help people answer questions. Things like Learn to Rank (LTR) become the most effective use of ML for Slack.
- You can get pretty far with rules. Use machine learning when that’s all that’s left.
- When you start applying observability to your data pipeline, a key lesson for Slack was to really focus on structured data, tracing, high cardinality events. This let them really use the tools they were already familiar with (ELK, Prometheus, Grafana) and go deep into understanding what’s happening in the systems.More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2PsVA4q
You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVqSubscribe: www.youtube.com/infoq
Like InfoQ on Facebook: bit.ly/2jmlyG8
Follow on Twitter: twitter… -
Bryan Liles on Making Kubernetes Easier for Developers, the CNCF, and “Serverless”
15 november 2019 · 30 min
Beluister deze afleveringIn this podcast, Daniel Bryant sat down with Bryan Liles, senior staff engineer at VMware. Topics covered included: the challenges with deploying applications into Kubernetes, using the open source Octant tool to increase a user’s understanding of Kubernetes clusters, and how “serverless” technologies may influence the future approaches to building software. Lees meer
Why listen to this podcast:
- Octant is a highly extensible platform for developers to better understand the complexity of Kubernetes clusters. Octant runs locally, using the local Kubernetes credentials. It currently displays information about a Kubernetes cluster and related applications as a web page. Soon this tool and resulting display will be provided as a standalone application.
- The goal of Octant is to enable users to discover what they need to discover. The tool aims to provide context relevant to where a user is and what they are trying to achieve. The Octant plugin system allows integration with other tooling, such as logging and metrics frameworks. This aims to facilitate quick problem detection and resolution.
- Cloud native platforms like Kubernetes are complicated, as there are lots of moving parts. The most important challenge to be tackled to increase the adoption of platforms like Kubernetes is “how do we move code from our IDEs to wherever it needs to run with the least amount of friction?”. Testing needs to be implicit, as does security verification, and the acts of deployment. Kubernetes needs its “Ruby on Rails” moment.
- Creating “serverless” systems is an interesting approach, but we may currently be using this technology in a non-optimal way. For example, creating web applications using this technology enables scalability, but can lead to the creation of difficult to understand systems that also require a lot of boilerplate configuration. Arguably, a more interesting use case is implementing large-scale batch processing using simple event-driven models.
- The Cloud Native Computing … -
Victor Dibia on TensorFlow.js and Building Machine Learning Models with JavaScript
8 november 2019 · 28 min
Beluister deze afleveringVictor Dibia is a Research Engineer with Cloudera’s Fast Forward Labs. On today’s podcast, Wes and Victor talk about the realities of building machine learning in the browser. The two discuss the capabilities, limitations, process, and realities around using TensorFlow.js. The two wrap discussing techniques like Model distillation that may enable machine learning models to be deployed in smaller footprints like serverless.
- While there are limitations in running machine learning processes in a resource constrained environment like the browser, there are tools like TensorFlow.js that make it worthwhile. One powerful use case is the ability to protect the privacy of a user base while still making recommendations.
TensorFlow.js takes advantage of the WebGL library for its more computational intense operations.
- TensorFlow.js enables workflows for training and scoring models (doing inference) purely online, by importing a model built offline with more tradition Python tools, and a hybrid approach that builds offline and finetunes online.
To build an offline model, you can build a model with TensorFlow Python (perhaps using a GPU cluster). The model can be exported into the TensorFlow SaveModel Format (or the Keras Model Format) and then converted with TensorFlow.js into the TensorFlow Web Model Format. At that point, the can be directly imported into your JavaScript.
- TensorFlow Hub is a library for the publication, discovery, and consumption of reusable parts of machine learning models and was made available by the Google AI team. It can give developers a quick jumpstart into using trained models.
- Model compression promises to make models small enough to run in places we couldn’t run models before. Model distillation is a process where a smaller model is trained to replicate the behavior of a larger one. In one case, BERT (a library almost 500MB in size) was distilled to about 7MB (almost 60x compression). Lees meerMore on this: Quick scan our curated show notes on InfoQ…
-
Michelle Krejci on Moving to Microservices: Visualising Technical Debt, Kubernetes, and GraphQL
1 november 2019 · 34 min
Beluister deze afleveringIn this podcast, Daniel Bryant spoke to Michelle Krejci, service engineer lead at Pantheon, about the Drupal and Wordpress webops-based company’s move to a microservices architecture. Michelle is a well-known conference speaker in the space of technical leadership and continuous integration, and she shared her lessons learned over the past four years of the migration. Lees meer
Why listen to this podcast:
- The backend for the Pantheon webops platform began as a Python-based monolith with a Cassandra data store. This architecture choice initially enabled rapid feature development as the company searched for product/market fit. However, as the company found success and began scaling their engineering teams, the ability to add new functionality rapidly to the monolith became challenging.
- Conceptual debt and technical debt greatly impact the ability to add new features to an application. Moving to microservices does not eliminate either of these forms of debt, but use of this architectural pattern can make it easier to identify and manage the debt, for example by creating well-defined APIs and boundaries between modules.
- Technical debt -- and the associated engineering toil -- is real debt, with a dollar value, and should be tracked and made visible to everyone.
Establishing “quick wins” during the early stages of the migration towards microservices was essential. Building new business-focused services using asynchronous “fire and forget” event-driven integrations with the monolith helped greatly with this goal.
- Using containers and Kubernetes provided the foundations for rapidly deploying, releasing, and rolling back new versions of a service. Running multiple Kubernetes namespaces also allowed engineers to clone the production namespace and environment (without data) and perform development and testing within an individually owned sandboxed namespace.
- Using the Apollo GraphQL platform allowed schema-first development. Frontend and backend teams collaborated on creat… -
Ryan Kitchens on Learning from Incidents at Netflix, the Role of SRE, and Sociotechnical Systems
4 oktober 2019 · 29 min
Beluister deze afleveringIn today’s podcast we sit down with Ryan Kitchens, a senior site reliability engineer and member of the CORE team at Netflix. This team is responsible for the entire lifecycle of incident management at Netflix, from incident response to memorialising an issue. Lees meer
Why listen to this podcast:
- Top level metrics can be used as a proxy for user experience, and can be used to determine that issue should be alerted on an investigated. For example, at Netflix if the customer playback initiation “streams per second” metric declines rapidly, this may be an indication that something has broken.
- Focusing on how things go right can provide valuable insight into the resilience within your system e.g. what are people doing everyday that helps us overcome incidents. Finding sources of resilience is somewhat “the story of the incident you didn’t have”.
- When conducting an incident postmortem, simply reconstructing an incident is often not sufficient to determine what needs to be fixed; there is no root cause with complex socio-technical systems as found at Netflix and most modern web-based organisations. Instead, teams must dig a little deeper, and look for what went well, what contributed to the problem, and where are the recurring patterns.
- Resilience engineering is a multidisciplinary field that was established in the early 2000s, and the associated community that has emerged is both academic and deeply practical. Although much resilience engineering focuses on domains such as aviation, surgery and military agencies, there is much overlap with the domain of software engineering.
- Make sure that support staff within an organisation have a feedback loop into the product team, as these people providing support often know where all of the hidden problems are, the nuances of the systems, and the workarounds.More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2LLwk8T
You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest to… -
Oliver Gould on the Three Pillars of Service Mesh, SMI, and Making Technology Bets
20 september 2019 · 25 min
Beluister deze afleveringIn this podcast we sit down with Oliver Gould, co-founder and CTO of Buoyant. Oliver has a strong background in networking, architecture and observability, and worked on solving associated technical challenges at both Yahoo! and Twitter. Oliver is a regular presenter at cloud and infrastructure conferences, and alongside his co-founder William Morgan, you can often find them in the hallway track, waxing lyrical about service mesh -- a term they practically coined -- and trying to bring others along on the journey. Lees meer
Service mesh technology is still young, and the ecosystem is still very much a work in progress, but there have been several recent interesting developments within this space. One of these was the announcement of the service mesh interface (SMI) at the recent KubeCon EU in Barcelona. The SMI spec seeks to unlock service mesh integrators and implementers, as this can provide an abstraction that removes the need to bet on any single service mesh implementation. This can be good for both tool makers and enterprise early adopters. Many organisations like Microsoft and HashiCorp are involved with working alongside the community to help define the SMI, including Buoyant.
In this podcast we summarise the evolution of the service mesh concept, with a focus on the three pillars: visibility, security, and reliability. We explore the new traffic “tap” feature within Linkerd that allows near real time in-situ querying of metrics, and discuss how to implement network security by leveraging the primitives like Service Account provided by Kubernetes. We also discuss how reliability features, such as retries, time outs, and circuit-breakers are becoming table stakes for infrastructure platforms.
We also cover the evolution of the service mesh interface, explore how service meses may impact development and platforms in the future, and briefly discuss some of the benefits offered by the Rust language in relation to building a data plane for Linkerd. We conclude the podc…
-
Event Sourcing: Bernd Rücker on Architecting for Scale
13 september 2019 · 25 min
Beluister deze afleveringToday on the podcast, Bernd Rucker of Camunda talks about event sourcing. In particular, Wes and Bernd discuss thoughts around scalability, events, commands, consensus, and the orchestration engines Camunda implemented. This podcast is a primer on considerations between an RDBMS and event-driven systems. Lees meer
Why listen to this podcast:
- An event-driven system is a more modern approach to building highly scalable systems.
- An RDBMS system can limit throughput in scalability. Camunda was able to achieve higher levels of scale by implementing an event-driven system.
- Command and events are often confused. Commands are actions that request something to happen. Events describe something that happened. Confusing the two causes confusion in application development of event-driven systems. -
Pat Kua on Technical Leadership, Cultivating Culture, and Career Growth
6 september 2019 · 27 min
Beluister deze afleveringIn this podcast we discuss a holistic approach to technical leadership, and Pat provides guidance on everything from defining target operating models, cultivating culture, and supporting people in developing the career they would like. There are a bunch of great stories, several book recommendations, and additional resources to follow up on. Lees meer
* Cultivating organisational culture is much like gardening: you can’t force things, but you can set the right conditions for growth. The most effective strategy is to communicate the vision and goals, lead the people, and manage the systems and organisational structure.
* N26, a challenger bank based in Berlin has experienced hypergrowth over the past two years. Both the number of customers and the amount of employees have increased over threefold. This provides lots of opportunities for ownership of product and projects, and it creates unique leadership challenges.
* A target operating model (TOM) is a blueprint of a firm's business vision that aligns operating capacities and strategic objectives and provides an overview of the core business capabilities, internal factors, and external drivers, strategic and operational levers. This should be shared widely within an organisation
* Pat has curated a “trident operating” model for employee growth. In addition to the class individual contributor (IC) and management tracks, he believes that a third “technical leadership” track provides many benefits.
* People can switch between these tracks as their personal goals change. However, this switch can be challenging, and an organisation must support any transition with effective training.
* Pat recommends the following books for engineers looking to make the transition to leadership: The Manager’s Path, by Camille Fournier; Resilient Management, by Lara Hogan; Elegant Puzzle, by Will Larson; and Leading Snowflakes by Oren Ellenbogen. Pat has also written his own book, Talking with Tech Leads.
* It is valuable to define organisation va… -
Thomas Graf on Cilium, the 1.6 Release, eBPF Security, & the Road Ahead
2 september 2019 · 27 min
Beluister deze afleveringCilium is open source software for transparently securing the network connectivity between application services deployed using Linux container management platforms like Docker and Kubernetes. It is a CNI plugin that offers layer 7 features typically seen with a service mesh. On this week’s podcast, Thomas Graf (one of the maintainers of Cilium and co-founder of Isovalent) discusses the recent 1.6 release, some of the security questions/concerns around eBPF, and the future roadmap for the project. Lees meer
Why listen to this podcast:
* Cilium brings eBPF to the Cloud Native World. It works across both layer 4 and a layer 7. While it started as a pure eBPF plugin, they discovered that just caring about ports was not enough from a security perspective.
* Cilium went 1.0 about a year and a half ago. 1.6 is the most featured-packed release of Cilium yet. Today, it has around 100 contributors.
* While Cilium can make it much easier to manage IPTables, Cilium overlaps with a service mesh in that it can do things like understand application protocols, HTTP routes, or even restrict access to specific tables in data stores.
* Cilium provides both in kernel and sidecar deployments. For sidecar deployments, it can work with Envoy to switch between kernel space and userspace code. The focus is on flexibility, performance, and low overhead.
* BPF (Berkeley Packet Filter) was initial designed to do filtering on data links. eBPF has the same roots but it’s now used for system call filtering, tracing, sandbox, etc. It’s grown to be a general-purpose programming language to extend the Linux kernel.
* Cilium has a multi-cluster feature built-in. The 1.6 release can run in a kube-proxy free configuration. It allows fine-grain network policies to run across multiple clusters without the use of IPTables.
* Cilium offers on-the-wire encryption using in-kernel encryption technology that enables mTLS across all traffic in your service fleet. The encryption is completely transparent to the applica… -
Yuri Shkuro on Tracing Distributed Systems Using Jaeger
28 augustus 2019 · 32 min
Beluister deze afleveringThe three pillars of observability are logs, metrics, and tracing. Most teams are able to handle logs and metrics, while proper tracing can still be a challenge. On this podcast, we talk with Yuri Shkuro, the creator of Jaeger, author of the book Mastering Distributed Tracing, and a software engineer at Uber, about how the Jaeger tracing backend implements the OpenTracing API to handle distributed tracing. Lees meer
Why listen to the podcast:
- Jaeger is an open-source tracing backend, developed at Uber. It also has a collection of libraries that implement the OpenTracing API.
- At a high level, Jaeger is very similar to Zipkin, but Jaeger has features not available in Zipkin, including adaptive sampling and advanced visualization tools in the UI.
- Tracing is less expensive than logging because data is sampled. It also gives you a complete view of the system. You can see a macro view of the transaction, and how it interacted with dozens of microservices, while still being able to drill down into the details of one service.
- If you have only a handful of services, you can probably get away with logging and metrics, but once the complexity increases to dozens, hundreds, or thousands of microservices, you must have tracing.
- Tracing does not work with a black box approach to the application. You can't simply use a service mesh then add a tracing framework. You need correlation between a single request and all the subsequent requests that it generates. A service mesh still relies on the underlying components handling that correlation.More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2ZlvMyR
You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVqSubscribe: www.youtube.com/infoq
Like InfoQ on Facebook: bit.ly/2jmlyG8
Follow on Twitter: twitter.com/InfoQ
Follow on LinkedIn: www.linkedin.com/company/infoq
Check the landing page on InfoQ: https://bit.ly/2ZlvMyR -
Louise Poubel on the Robotic Operating System
19 augustus 2019 · 29 min
Beluister deze afleveringROS is the Robotic Operating System. It’s been used by thousands of developers to prototype and create a robotic application. ROS can be found on robotics in warehouses, self-driving car companies, and on the International Space Station. Louise Poubel is an engineer working with Open Robotics. Today on the podcast, she talks about what it takes to develop software that moves in physical space, including the Sense, Think, Act Cycle, the developer experience, and architecture of ROS. Lees meer
Why listen to this podcast:
- Writing code for robot development, you use the Sense, Think, Act Cycle.
- ROS is an SDK for robotics. It provides a communication layer that enables data to flow between nodes that handle sensors, logic, and actuation.
- ROS has two versions and has been around for twelve years. ROS 1 was entirely implemented in C. ROS 2 offers is a common C layer with implementations in many different languages, including Java, JavaScript, and Rust.
- Released on a six-month cadence, Dashing was the latest release (May 2019). Previous releases were supported for one year, Dashing is the first LTS and will be supported for two years.
- ROS 2 builds on top of the standard Data Distribution Service (DDS) that you find in mission-critical systems like nuclear power and airplanes.
- Simulation is an important step in robotics. It allows you to prototype a system before deploying to a physical system.
- Rviz is a three-dimensional visualizer used to visualize robots, the environments they work in, and sensor data. It is a highly configurable tool, with many different types of visualizations and plugins. It allows you to put together all your data in one place and see it.More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2ZbOvrO
You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVqSubscribe: www.youtube.com/infoq
Like InfoQ on Facebook: bit.ly/2jmlyG8
Follow … -
Matt Klein on Envoy Mobile, Platform Complexity, and a Universal Data Plane API for Proxies
9 augustus 2019 · 41 min
Beluister deze afleveringIn this podcast we sit down with Matt Klein, software plumber at Lyft and creator of Envoy, and discuss topics including the continued evolution of the popular proxy, the strength of the open source Envoy community, and the value of creating and implementing standards throughout the technology stack. We also explore the larger topic of cloud natives platforms, and discuss the tradeoffs between using a simple and opinionated platform against something that is bespoke and more configurable, but also more complex. Related to this, Matt shares his thoughts on when and how to make the decision within an organisation to embrace technology like container orchestration and service meshes. Lees meer
Finally, we explore the creation of the new Envoy Mobile project. The goal of this project is to expand the capabilities provided by Envoy all the way out to mobile devices powered by Android and iOS. For example, most current user-focused traffic shifting that is conducted at the edge is implemented with coarse-grained approaches via by BGP and DNS, and using something like Envoy within mobile app networking stacks should allow finer-grained control.
Why listen to this podcast:
- The Envoy Proxy community has grown from strength-to-strength over the last year, from the inaugural EnvoyCon that ran alongside KubeCon NA 2018, to the increasing number of code contributions from engineers working across the industry
- Attempting to create a community-driven “universal proxy data plane” with clearly defined APIs, like Envoy’s XDS API, has allowed vendors to collaborate on a shared abstraction while still allowing room for “differentiated success” to be built on top of this standard
Google’s gRPC framework is adopting the Envoy XDS APIs, as this will allow both Envoy and gRPC instances to be operated via a single control plane, for example, Google Cloud Platform’s Traffic Director service.
- There is a tendency within the software development industry to fetishise architectures that are desi… -
Armon Dadgar on HashiCorp Research, the Evolution of Infrastructure Tooling, and Standardisation
2 augustus 2019 · 23 min
Beluister deze afleveringOn this podcast, we’re talking to Armon Dadgar, co-founder and CTO of HashiCorp. Alongside Mitchell Hashimoto, Armon founded HashiCorp over six years ago, and the company has gone from strength to strength, with their open source infrastructure product suite now consisting of Consul, Nomad, Vault and Terraform. Lees meer
We discuss the formation of the HashiCorp research division, and explore some of the computer science research underpinning Consul and Nomad. We also cover the challenges of supporting teams when they are looking to embrace new modes of working with dynamic infrastructure, and Armon introduces the new learn.hashicorp.com educational website and accompanying community and support forums.
Why listen to this podcast:
- There is a lot of fundamental computer science research that underpins the HashiCorp infrastructure workflow and configuration tooling. This helps to ensure that these mission-critical tools perform as expected, and enables sound reasoning about scaling these technologies.
- The HashiCorp founders recognised the value of creating an industrial research-focused department within the company even when there were only 30 staff.
- The Consul service mesh and distributed key value store leverages consensus and gossip algorithms from computer science research, Raft and SWIM, respectively. The HashiCorp team contributed a novel research-based improvement to SWIM -- Lifeguard: SWIM-ing with Situational Awareness -- that was presented at the DSN academic conference
- Initially HashiCorp produced a new tool every 6-12 months, focusing on filling gaps within the infrastructure workflow tooling market. Now the focus is on refining the operator/user experience of the existing tools, creating more integrations with other platforms and tooling, and facilitating engineering teams adopting these tools, via the creation of educational resources and community forums.
- Standardisation within computing technology can offer many benefits, especially where interope…