以下是转录的英文原文，为了流畅阅读，部分语句稍有删减。本次分享所用 PPT 与网络上之前已有的一些 PPT (opens new window) 几乎一致，可以直接对照查看。
Yeah, I'm going to talk about Deno. It's a project I've been working on for the last year and a half with some people. Just out of curiosity, have you guys heard of Deno? A couple of you. Most of you. That's good to hear. Before I start, I just need to make a small disclaimer, you know, Deno is a project that's very similar to Node. It's something like my second iteration on it. And often, this kind of causes people to panic because they've invested a lot of time into learning Node and they have a lot of software that runs on Node, and they might worry that, they're going to need to rewrite all of that software. It causes panic for some people. And I just want to assure you that Node is very stable software and it has a very slow and steady release cycle. Node is going to be on the ground for a long time. This project is about exploring new ideas in the same space without disrupting existing software. So don't panic. This is meant for people interested in exploring what else is possible, what else can we do with in this space.
So I think what we're trying to do with this project is we think that dynamic languages are very useful things, right? A lot of people, you know, Rust is around and Go is around, and these are very good systems for certain problems, but I think there're a lot of problems that are best dealt with dynamic languages. If you need to rename a bunch of files in a directory, or you want to write a prototype very quickly, you want a scripting language. So, we think that not enough effort is being put into improving dynamic language platforms. And this project is an attempt to make this really nice. So, it's a fun and productive scripting language. I think neither Python nor Ruby nor Node are completely satisfactory systems, and it's an important enough problem that we should give some effort to.
So anyway, we use TypeScript. And kind of the interesting thing with Deno is this idea of our package manager. I said that it doesn't rely on a single server. There's no fixed server. The way that this works is rather than running a program like NPM to fetch code from some system, you can just put the URL directly into your code. And you know, Deno knows how to go down and fetch that URL, cache it onto the hard drive, and link that into your program. What's interesting about this is that this is how our websites work. So, this is kind of standards-compatible way of linking websites.
If you use ES modules and you do an
import statement with a URL, this will work in Chrome. So, hopefully we're providing a very simple but usable way of linking to third party code. The way I think of it is you have your code, which is local on your machine, your project, that's on your hard drive. And you would link to that code using relative URLs like
./foo.js. But once you want to link to a third-party module, that would be over HTTP somehow. I'm going to demonstrate this in a second, but this code follows the same semantics as web browsers, so when it's downloaded, it stays on your hard drive and it's never flushed. The point is that when you're on an airplane or you're in China and can't access the Internet, you can still run these programs because they're cached on your hard drive.
We're trying to be more browser compatible than that in Node, and notably, we kind of invented a lot of APIs that mirrored browser APIs, and in Deno, we're trying to be very careful to make where, you know, obviously we're doing stuff outside of the web browser, we're opening files and we're opening network connections, and a lot of these things don't have corresponding APIs in the web browser, but a lot of APIs do have corresponding. And more and more, there's a lot of web APIs that do this sort of things. So, we try to be as browser compatible as possible. For example, we have fetch built in, so if you want to make a new HTTP request, you can use the fetch API. And we try to keep that as close as possible to the browser. We have a set of standard modules. The idea is that, we all kind of need some modules to be to rely on in Node. We built some functionality into the Node executable, but you kind of have this situation where we were trying to keep the core small, but as a result, there was a lot of utility functions that people needed and they ended up getting those from third party sources.
And then what happens is, you know, because these are third party modules, and they're controlled by some untrusted entity. Some random person has some module there. This kind of introduces a security problem. This is an attack vector. Somebody can somehow take over an NPM package for some small utility module, and unless you're very good at checking all of the code that you're running, that can be a way to attack. Anyway, we're trying to do this differently, so we have a large standard module system and we review all of the code. You're not allowed to just randomly insert code into there, so this is secure code, and it's based closely on the go standard library. We kind of want to take advantage of the intellectual effort that went into the go project and kind of reuse a lot of the ideas from them, and yes, I mentioned it's secure by default, so without any flags, you are run inside of a sandbox. Without doing anything, you can't access the disk, you can't access the network, and the person who is executing the program needs to opt into these permissions. And then to opt into these permissions, there's various flags. So, you can do like
--allow-read, that will give you access to the file system, or you could say
--allow-env to, to access with environmental variables. With all that said, let me just type a little bit and kind of show you what I'm talking about.
You can execute a REPL just by typing
hello world example. You can do
deno run, and that executes
hello world. I mentioned that this is running inside of a secure sandbox, right? So, what happens if we try to access the disk? Let's try to open a file now. I've got this file,
hello.txt, which is
Deno.open('hello.txt'). This will open a file, and everything's using async await, so we need to give it await there. And, we get a file object. And just to make sure this is what I think it is, let's just print that out. So, let's do
deno run hello_world.js. And now I've got this error. It says, uncaught permission denied, run again with
--allow-read flag because I was trying to open something from the disk. So, this is what I mean by a bit of security. We don't allow you to randomly open files from the disk.
Now I was able to open the file and I've printed it out. Let's try to improve this a little bit. What we're going to do is do this
Deno.copy(), and this file object is like a stream. Maybe this
hello.txt is 10GB or something, a very large file. We don't necessarily want to buffer all of that file in memory, so this file object is just a file handle. What we want to do is now stream this to stdout. So, we want to read a little bit of data from the file, right into stdout. The way we do this is the
Deno.copy() function and the destination is
Deno.stdout and the source is file. This is doing a little loop of copying data from the file to stdout, and we're going to await that, and maybe at the end I'll do
console.log and just so we know that we're done. Okay, so let's try that again. So now we've printed
hello world out, right?
I mentioned that we can load files from URLs. Let's see if this works actually. Exactly, my Internet situation here is a little sketchy. So this URL, if you can see it as a URL to a file, and if I load this in a web browser, you know, we have the Deno land website and we kind of see the text that this source code, this is some HTML, obviously. We have this kind of fancy trick in here that if you grab this URL and you access it from
curl, then you get the raw source code rather than the HTML version of this. This is just a stupid little trick, but there's exact header and web browsers, we can use that to detect whether the client is trying to display a website or is asking for some raw data. So, if I just
curl that, hopefully I have the raw contents here, and it's basically grabbing the command line arguments. It's an array of file names. This is supposed to be like the cat program. So, the way that I do
cat hello.txt, what I want to do is do it a similar thing with Deno. Basically, we're doing the same thing that we did in the previous program. We're just looping through each of the files and we're opening them, and then we're copying them to stdout. Very simple, but just to demonstrate how we run programs that are on the Internet, so this program is hosted through HTTP. Let me try to
cat hello.txt. As you see, again, I get this permission denied, obviously just because it's a third-party module, doesn't mean that it has permission to access the disk, so I need to
--allow-read here, or there's a little trick if you do
-A, you just opt out of everything, so this is just allow all things; never give me a permission denied, which, you know, during development is usually what you want. So, this thing works as expected.
And the question is, this seems to run very fast, like it doesn't seem to be making a network connection as I'm running this thing. I've mentioned before that the URLs are cached, and what you can do is you can examine using
deno info, and hopefully it's going to tell you some information about that URL and where it's cached on the hard drive. If I run this, it says that the local version of it is
This program itself is not so interesting, because it doesn't have any dependencies. But more complicated programs are going to import other things. Here's a more complicated program,
http/file_server.ts. This program runs a local web server on your computer and hosts a local file through file server. Let's try running that now. It says permission denied. You know, it's trying to start a server. You can't do that without network access, so I need to do
--allow-net and now I've got a server running. If I tried to
curl this server here, how does that work? All right, I got an internal server error. Why do you think that is? I was actually expecting this error out up here, but it didn't. It might be a bug. But yeah, we haven't given it file access, so it's not able to serve the local directory because it hasn't been given access to the local disk. So, what we need to do is give it
--allow-read here. So now when I
curl this, I should get some HTML, so now I know I'm serving up the local directory, right? So, I can see like that file that we just made. So, just as an example of a more complicated program, let's use the info with this more complicated program now. So, this thing obviously has a lot of dependencies. And what you see here, you know, along with the kind of cached file stuff. You get a tree of dependency of all of the modules that it's using, because we've built this on top of web APIs, this sort of stuff, and other tooling inside of Deno is usable for code that's just used on websites. If you're using ES modules and you're not using Deno at all, but you want to say, examine the dependency, the tree of your program, this should work for you.
So, let me do it one more small demo. We have a bunch of tooling inside of this, and if you do
deno help, you'll see a bunch of things. I'm not going to go into all of this stuff. It's too much. But when I do want to show you is this one feature called
deno bundle and it allow you to take a program, all of the dependencies, which are many files, which might be on an HTTP server, might be locally on your disk, bundle those all into a single thing. It's just like Webpack, right? So, if I do this,
deno bundle. What we should do is get some stuff printed to stdout, which is all of the code inside the file server. And you know, what you can do is pipe that to a file, and if I do like
For the future, we're working very hard to ship a stable release. We've been shipping beta releases for the last year. We hope to have 1.0 out probably this month, maybe January. You know, it's hard. But yeah, we're very interested in using WebAssembly or kind of providing users with a way to, you know, import Rust code, and do this in a secure way somehow. We're also interested in doing a WebGL bindings, which would enable users to do things like TensorFlow.js, to do accelerated matrix multiplication. WebCrypto is another, often asked for, web API. So, there's kind of these web APIs that are clearly target audience, and it would be very nice to have a single executable that you don't need to install Cuda for, and can suddenly do a fast matrix multiplication. This enables very fast development time for that sort of projects.
To the rest of it. I would just say this is an open collaboration we have. It's not just me working on it. We have many contributors. And here's my email address. Feel free to email me if you have any questions or just come up and talk to me. And the website, obviously.
I don't know what our timing situation is here. Is there time for questions?
Q1: I just wonder if you import the URL, and you cache it, how can you update when it is updated?
So, what happens so that URL might change, obviously. Right? If we cache this forever, how do you get the updates? It's just like in the web browser, you know; you've cached some HTML, and if you hit reload, you're just getting the same cache each HTML, but if you hit shift and then reload and then it will re-download this stuff. I'll show you an example of that. The way that you do this, say where we're running this file server, right? We're trying to run this file server. Maybe this URL is getting updated in the last couple of minutes and we want to get that new code. We just do a --reload, and then this is going to re-download all of the dependencies for that. So, it's based off of web browser semantics.
Q2: In Node.js we have dependencies, and, it's a problem that we have different versions of dependencies, is there any version of TypeScript files on the net?
In NPM you can specify versions and there's different versions of modules, but I haven't mentioned anything about versions here. What I would say is that when you
import an NPM module, say, you are using express. There's a string
"express", and that specifies the module. Implicit to that is the NPM server, and then implicit to that is that you get the latest version of express. Now you can also do
"email@example.com", or whatever, and specify a version up to that. So now you have a string that specifies that module name and the version. And then implicit to that is the server, right? NPM is the server. So that's implicit. All of this can be factored into the URL. So, this is just a string, right? What we can do is put the version into the string. And for Deno land, we've come up with a certain scheme to do this, so what you can do is like @v0.5.0, this will specify a specific version of that URL. So, by putting the version in the URL itself, I think you're fully specifying this now, of course, this is still up to the server to show the same version every time. Right. But I would say that's still up to NPM to still ship you the same version of express, right?
I mean, who knows? They could send you something else. What we're doing here is we're just explicitly specifying the server, and there may be a Deno package repository that makes the promise that every time you go to a URL, you get exactly the same file, but that's up to the server. We're agnostic to that, just like the web, but I think the version problem can be solved by specifying versions in the URL.
Q3: Will Deno prevent the cache folder being too large?
No, not currently. I mean, it should, that should be a feature. We don't have that feature yet. So basically, what we want is an LRU, like purging the lease recently used to have like some upper size limit of on the cache, say 500MB. You know, after that we should purge some files. We don't have that feature in there, but it's certainly going to be something that you think, yeah.
Q4: If you
importa URL in the file, as in Go, they have made a module management system. So, does Deno has this plan for a centralized dependency management?
Nope. We're trying to keep it simple and trying to keep it very browser centric, staying with the same things that the browser is specified, and then add tooling, like this bundling thing. So, users who need, say, vendor dependencies can do so. But in terms of having a centralized thing, we have the Deno land server, which, you know, in some sense as a centralized system. But beyond that, no.
Q5: Does Deno support something like mirrors, because you know, in China there is no Internet.
Yeah, I can't remember. So, people are asking for this, and I'm just going to check the documentation. Yeah. I think we have HTTP_PROXY, so you can set it, probably contributed by a Chinese user. I think this is a recent issue, so this is being worked on, so you know, maybe.
Q6: You have mentioned that it's up to the server to manage the versioning thing, and will Deno provide a standard library to create a repository for versioning or something?
My point is that this is a fairly complicated code base with this, and it's actually a separate project. It's called demo website 2; this website is written in Node currently. The problem is that we have kind of a chicken and egg sort of problem. So, we're kind of trying to build this system, but we also need the server at the same time. So, making a dependency on Deno is too difficult for us right now. So, this part is written in Node. Actually, I would very much like to have this ported over to Deno and then put in as kind of a standard module so that people can run their own Deno land. But. It's going to take a while before we can actually run this code base.
Q7: When do you think is time you to really put this into production use, maybe just for some internal projects?
I think when we hit 1.0, that's it. That's basically going to mean like we'll make some stability guarantees about APIs. Currently we're changing APIs without any notice. So, once 1.0 comes out, which as I said, Yes.
Q8: How about Deno with machine learning? Are there more details?
Yeah, I'm very interested in this topic. So, TensorFlow.js I think is a spectacular project, and TensorFlow.js uses WebGL, right? In TensorFlow.js, there's actually two backends right now. There's one for running on Node, and there's one for running in the web browser. And running on Node uses TensorFlow and uses kind of TensorFlow's Cuda libraries. In web browser backend there's this WebGL thing. What I'm hoping to do is to basically support WebGL, and then we can run these machine learning codes on top of that. This was actually like one of the very first things that I wanted to do with this system, but it's proven difficult to get all of these smaller modules loaded, and kind of the correctness of the system is more important. So, we're basically trying to get the fundamentals working before we start adding on new things. But yes, WebGL. I want to support it.
Q9: I still have a question about the version controlling, In something like semantic version control, when we just want to stick a package to a major version, how can we express that in a URL?
So, in semantic versioning, what you might want to specify is I just want version 1.2 and I want to be able to take any patch updates that are coming through. I don't want to tie myself to a specific version. So, the way this would work in Deno is up to the server just as before. When you specify the version, say this is a full semantic version, you know, maybe you want to link to a code that, you know, potentially starts upgrading underneath you, right? Cause you want to get, bug fixes. What you would like to do is just link to something like this without that, or maybe even something like this, some, some way of doing "^1.2.0". This doesn't work. The Deno land server does not support this. How Deno land server works is that it goes to GitHub and gets the content, so what we allow is linking to branches. The idea of this is to kind of link to branches, and so you could potentially do something like that. You know, "@stable" where that's a branch, and then you'll always kind of get that stable code base, which you know, may come with bug fixes if you do --reload. So, it's up to the server basically.
Q10: When we first
importa TS file, we might not specify the version, but if, say, my teammates joined our team and downloaded the packages, their version can be different. Is there a lock file for this?
There is, so,
deno help run, we have this lock file feature. So yes, you can, if you're in CI or whatever, you can fix the shock in some of the file.
Q11: How do you compare between the Node.js and Deno?
So, this is a
hello world web server that's just seeing how much throughput you can do on a
hello world server, so the items to look at here is Node TCP and Deno TCP, and bigger is better. So, Node TCP is 51,000 requests per second, and Deno TCP is 41,000. So, my point is that we know, and we're working on it, we're just trying to kind of, you know, ladder up through this system.
I haven't mentioned how kind of the internal details of this, but the internals of Deno function much differently than Node. Maybe I won't go into this now, but there's basically an abstraction layer problem where all calls out of the VM go through a central bottle now, right? So we kind of have something like a "syscall" in Deno, whereas in Node, we kind of make separate bindings for every API. And we're kind of funneling everything through this one bottleneck. It's a good thing because we're going to have very uniform performance for different API because they all go through the same system.
But kind of the bad part of this is that we're not able to optimize individual APIs directly, anyway. My partner and I are kind of insanely into performance. We don't think that this is the most important problem right now. I very much hope that we will address this in the near future.
Q12: I want to ask, there are lot of popular packages in the Node ecosystem. Does Deno have any plan for supporting these packages?
So, compatibility with existing Node things. Early on this, this project started as an experiment to see how we could do things differently and we didn't want to start with Node compatibility because that would force too many design decisions. At this point, our software is fairly mature, and we've got kind of art abstractions worked out, and so we've started a new project. It's part of the standard module. It's called
std/node. And here we're starting to fill in the Node APIs and make that this bigger. So, for example, we have a file system (fs). This project has just started a couple of weeks ago, but basically what we hope to do is have kind of, an API-compatible standard module for Deno. And in this way, we can start using Node code, hopefully. In particular, let me point out, we have, global. We have a require implementation oh no, it's module. Let me just go into the documentation. So basically, you can create a require and you can require FSA. So anyway, the short answer to your question is that we're working on it. We hope, you know, now that we've kind of established our abstractions, we hope that this will be better, but currently it's operating.
Q13: As developers we're concerned about security. How do you guarantee that the third-party code with malware cannot escape from the sandbox?
So the question is, you know, you've got some third party code, but maybe there's some intermediate party that has injected some bad code into this request, so you're making this HTTP request to the server and what you think is good code and you get back code. So, the answer to that is locked files. So, what you want to do is you want to develop your program and audit the code that you use. You need to trust the code that you use because it's in your program, right? So at least like at some point, you need to know what you're running, right? And then you create a lock file and then you can fix at those versions, right? So that they can update underneath you, but it will error if it doesn't match the same checksum as when you developed it.
Q14: Have you found any issues when third-party code escapes from the sandbox?
Yeah, there's tons of issues like that. I mean, this is very under development, and I don't doubt that there's many ways to break out the sandbox. The point is that those are bugs and we correct them. So instead of in Python and Ruby and Node, this is not a bug, this is normal operation. Right? But you know, it's going to take a while for us to mature and work out all of these many ways of things breaking out of the sandbox. But, yeah, I mean, more or less it's working, I would say. I wouldn't stake my life on it. I think this kind of comes over time, like as exploits are found and you fix them, and you know, add a test out of them. So, I think over time it becomes more secure.
Q15: Did you ever encounter any problem with the version of TypeScript? TypeScript is under development and there must be some issues.
We bundled TypeScript, so we have a fixed version of TypeScript, which you can see if I do Deno. And Deno is kind of where we put all of our non-webby things. And if I do Deno.version, I can see my TypeScript version. So, this is the version of TypeScript that is bundled into Deno. And yes, TypeScript is changing quickly and it's going to get out of date, but I think TypeScript has kind of, you know, had a very fast development cycle and is starting to stabilize. And I think the APIs are not changing as much anymore. And of course, Deno is kind of also going very fast at development and starting to stabilize what we would hope for in, you know, like a year or so. These things kind of stabilize. You don't want the languages to always be changing. So, what I'm kind of hoping is that this matters less and less as time goes on. It matters a little bit right now, especially with like top level await and that sort of stuff. But in six months the version of TypeScript is not going to matter as much as it did like a year ago.