I'm looking forward to the rest of this series! Here are my thoughts so far:
(1) I think I basically agree with you re precautionary principle paralysis and many safety efforts being in fact counterproductive. I propose we do something like
--Divide risks into those for which iterative development works (i.e. problems that, if they happen, we'll notice and fix before major catastrophe occurs) and those for which it doesn't (i.e. problems such that, if they happen, we either plausibly won't notice or plausibly won't fix before major catastrophe occurs)
--and then have fairly permissive / move-fast-and-break-things policies for the former and more restrictive precautionary-principle-esque policies for the latter.
(2) So far this series has been long on questions and considerations, and short on answers and proposals. Fair enough I guess, but it would be nice to see some more of the latter IMO. What are your opinions about e.g. AGI timelines, takeoff speeds, AGI governance strategies, AGI alignment strategies, likely failure modes of likely-to-be-used AGI alignment strategies, etc.?
(3) The seven essays promised in this series don't seem to contain anything about misalignment, superintelligence, or loss-of-control. Just putting in a vote here in case you care what I think (I won't take offense if you don't) that I'd love to hear your thoughts on those topics!
(1) I agree, I think this makes sense; follows the idea that we should be willing to tolerate some degree of harm before regulating, unless the scale/severity makes this untolerable.
(2)/(3) Yes that's fair, my aim with these is more to bring to light some tricky questions rather than claim to have solutions, but will try to be a bit more opinionated. On AGI, take-off speeds, superintelligence, loss of control etc - I have a lot of thoughts here (and much uncertainty) but I'm keeping these for a separate, longer piece. This will take a bit more time/work though, but hopefully I can share more before fast take-off ;)
I'm looking forward to the rest of this series! Here are my thoughts so far:
(1) I think I basically agree with you re precautionary principle paralysis and many safety efforts being in fact counterproductive. I propose we do something like
--Divide risks into those for which iterative development works (i.e. problems that, if they happen, we'll notice and fix before major catastrophe occurs) and those for which it doesn't (i.e. problems such that, if they happen, we either plausibly won't notice or plausibly won't fix before major catastrophe occurs)
--and then have fairly permissive / move-fast-and-break-things policies for the former and more restrictive precautionary-principle-esque policies for the latter.
(2) So far this series has been long on questions and considerations, and short on answers and proposals. Fair enough I guess, but it would be nice to see some more of the latter IMO. What are your opinions about e.g. AGI timelines, takeoff speeds, AGI governance strategies, AGI alignment strategies, likely failure modes of likely-to-be-used AGI alignment strategies, etc.?
(3) The seven essays promised in this series don't seem to contain anything about misalignment, superintelligence, or loss-of-control. Just putting in a vote here in case you care what I think (I won't take offense if you don't) that I'd love to hear your thoughts on those topics!
Thank you!
(1) I agree, I think this makes sense; follows the idea that we should be willing to tolerate some degree of harm before regulating, unless the scale/severity makes this untolerable.
(2)/(3) Yes that's fair, my aim with these is more to bring to light some tricky questions rather than claim to have solutions, but will try to be a bit more opinionated. On AGI, take-off speeds, superintelligence, loss of control etc - I have a lot of thoughts here (and much uncertainty) but I'm keeping these for a separate, longer piece. This will take a bit more time/work though, but hopefully I can share more before fast take-off ;)