Daily Standup 14
- Meta’s 175-billion-parameter large language model release
- Other large language models from organizations like OpenAI, Cohere and Hugging Face
- Welcoming Vatsal Shah (CTO of Speak Ai) back from India
- Techstack in-person poker event in Toronto
- Psychedelic Diaries podcast recording with Ray Christian
#facebook #meta #opensource #languagegeneration #openai #gpt #gpt3 #dalle #techto #psychedelics #toronto
Resources
https://ai.facebook.com/blog/democratizing-access-to-large-scale-language-models-with-opt-175b/
https://www.eventbrite.com/e/techstackto-002-tickets-216697617407
https://podcasts.apple.com/us/podcast/psychedelic-diaries/id1575791128
https://www.tiktok.com/@psychedelic_diaries
Automated Transcription
Alright, hello, today right here you know what? I had a chance yesterday so Vatsal Shah, CTO of Speak Ai. He returned from India and we’ve made a wonderful sign. And we were supposed to hold it at the airport and. We didn’t make it to the airport in time. We were a little late, so basically he was waiting out there for us. It was the opposite of what we were supposed to be doing, and so we didn’t get the help hold up the sign. It was a it was a sad experience, but we’re excited to welcome. That’s the homie standing at the the place for a little bit while you get settled back in Canada and he went back to visit some family after a long time through the pandemic and and then with COVID, another outbreak that trip that I think was supposed to be maybe three months ended up being like seven months and so he’s back.
Glad to have him back and for me there’s a. There’s definitely an excitement and energy in in working together in in person that is hard to replicate. You know, a lot of companies. Obviously moving towards remote work, but I think there’s something really magical but working in together in person solving problems in person and that physical presence that physical energy starts to stimulate the brain in a lot of different ways. And sometimes it’s even just as simple as going on a walk with someone and what? What that can what, what that can do for your mind and and your problem solving. So welcome back.
That’s all great to have you here. A couple of things that I want to talk about today. Very interesting news and Facebook meta. Whatever the name is, releases access to this large language model and from my understanding it’s not completely open. There is. I don’t know if it’s an application, but researchers are basically able to access this in sort of an open source manner and understand a little bit more about how these systems work, and I think that has been one of the challenges with.
I think AI in general, but especially these large language models you just don’t know how they’re coming to the output that they’re they’re coming to, and so 175 billion parameter language models. So that means, I mean, it was trained on a lot of data. This matches what GT3 has done, and I think that was intentional and purposeful and and different ways that people are using these systems, but really becomes a sort of multi purpose when people consider almost sentient. AI did allows you to analyze text, extract things, but also generate net new text. That’s where we’re seeing things from Dolly. Come out from open AI where you’re actually able to generate images, and really this these large language models seem to be a big part of what the future of AI will be with, and there’s definitely some skepticism here, and I think, especially when a company like meta like Facebook releases something like this with a track record that they’ve had flags.
Were raised and what’s the true, you know, intention here, and we’ve had obviously Facebook as a platform. Many of the applications that is in their ecosystem be free, but there are consequences to that. And obviously, that was the advertising model that emerged through Facebook and has caused lots of problems. What happens here? And also, what does the future look like? So available, yes, academic researchers of government, civil society and academic organizations and then research laboratories pretraining modeled in the code to train to use them.
The message right from a I Facebook AI labs meta itself. And again this is sort of a very exciting time for AI and that at least you know people are excited about the effort that Meta has made here to do this, and we’ll see what comes. But overall, the ability to interact with these systems to understand them better allows for a better level of transparency and more open understanding of how this technology. Is being generated and then applied and then used and then what are the ramifications of that? And this is not the only company most people are familiar with open AI, their GPT now three model Codex which is doing and then there are work with Dali as well too from an image generation part and this has sort of this parallel which is they have they are private company they are. You are paying to use this now. Again difference with Facebook here to say this is for non commercial.
Use the way that you can access this model and mount it yourself and everything. But still there are questions around. The ramifications of a company building models like this and then releasing them, and then I’ve had the pleasure that fun to be able to play with some of these systems in early in their beta and you could plug in some information and get some pretty wack responses. Things that are uncorrelated, downright rude, downright scary. And I think that’s where we are as a as a society today. Interacting with this kind of technology is you are struggling to understand how you got this.
Output, and that’s something that people want to solve and and need to solve another company right here in Toronto, which is fantastic I think. Raised about $125 million million round recently with some pioneers in the transformer models. Also doing work here. And same, you know. I’m guessing the same challenges that they’re having hadn’t had the and the pleasure to interact with a couple people on their team and see a lot of amazing work that they’re doing. One of the ones that we talked about was.
They have this auto transcript editor which I think is an amazing system. Had some challenges with the amount of characters that you could then input and then correct. You had to sort of batch it all and then remerge it all together so it didn’t really make sense to move forward with, but then these questions of accuracy emerge or if they don’t have the signal of the audio or understanding of who the person speaking is they could make corrections that are inaccurate. So a lot of great applications of this kind of technology but. Lot of possible bias along the way. A lot of challenges along the way and something that again lots of people continuing to keep an eye on. Glad there are ethics. You know, people working on AI from an ethical perspective, how do you apply this? How do you train models in a non biased way and truthful truthfully, like the world we have even the access to language, we have the people and the population you are going to have bias in these models no matter what. So we’ve seen these challenges first hand as speak, aye?
Many of these companies, on a larger scale are facing the same thing. One of their last company I’ll highlight here quickly hugging face, which is hosting these models. So amazing company great like. Just love what they’re doing. Sort of open source AI making it a better, better access to these kind of technologies in this open source manner. And they’re sort of hosting, mounting, storing their own language models, and as companies release their their work their then putting it on hugging face.
Or other developers to access and use so very exciting work from all of these companies here applying in different ways. I think we’ll continue to see this proliferation of large language models, some private, some like what meta is done, where it’s like free for non commercial use and then we’ll see what happens with commercial use and then other companies or organizations or just communities as a whole building full open source availability of this kind of technology. So very fascinating stuff. If you have any insights. Any thoughts on this? Would love to. I still have to take a deeper dive.
On it and and see what’s going on here, but pretty pretty exciting stuff and something I’m definitely following a couple other quick notes here. This is going to be fun to this. An event tonight. This event has shifted the date probably three Times Now just because of COVID. I forget when we first signed up to go to it, but tech stack, associated with tech Tio pretty on Braden. Reem, who started voice flow, which is an incredible company building. Sort of conversational UI.
To interact with Alexa, to interact with Google, and really has become a leader in that space. He likes poker, wanted to put together the community for it and bring it together for an event, and it was lucky enough to get a ticket. You know what? I’m not lucky enough is that I’m not a very good poker player and I even had to watch a reminder video of myself yesterday. So I’m I’m I’m going to go in, try to not look like an idiot. I don’t know if we’ve been buffeted by skills, but we’ll we’ll see. Hopefully tomorrow. I’m not coming back. You know, with a report that I.
Harris myself on this poker table here, but overall exciting event. Great way to bring the community together. You know helping work the the brain mind generate some good conversations and have some fun. So looking forward to that. Appreciate Braden you putting that on and everyone who was involved in in making this a great event. So looking forward to to this a lot and it’s nice out which is great and then last thing. Just a quick note. I think this is sort of fun. I’m going on in a podcast today. Ray Christian who has actually done some really incredible.
Work in text analysis and audio and video analysis. Applying that to mental health and then now looking deeper into sort of psychedelics has released this podcast. Psychedelic Diaries and I’m on it today, so I’m going to record an episode and probably be posted sometime in the future, so I’ll share that out when I do, but he’s got some questions for me. I’m going to answer those questions a little bit more focused on psychedelics and then the actual AI space as well as just some fun conversation. There, so again, I’ll share that out when you do it and you know what he’s doing. A good job he’s blowing up on. He’s blowing up on Tik T.O.K here with a bunch of these, you know, just a bunch of content around psychedelic, sort of spiritualism. And you know, challenges with the space challenges with the the work that you do with the medicine and even sharing some clips from some of these podcast episodes. So maybe I’ll even see myself on one of these tik T.O.K clips. My own tick Tock journey is quite interesting.
And at some point I’ll share more on that. But man, it’s a whole other world. In there, I can tell you that. And one I’m both interested as being part of, but also quite quite, you know, I’ve been lost in the scroll a couple of times already, and I I just can’t can’t allow that to happen. So I think that’s everything for me today. A couple topics there. Large language models for meta. That’s all great to have you back Sir Tech stack to event playing some poker, hoping not to embarrass myself. And then lastly, the Psychedelic Diaries podcast.
By recording, so if there’s anything else that you’re thinking around, any of these topics always feel encouraged to to send me a message. Thanks again for checking this out. Hope you have a great rest of the day. Bye.
