Is there something as growing too quickly?

If your manager feels you are ready for it, I would pursue it! Like Alex mentioned, this is a great position to be in.

The thing to be concerned about when it comes with fast promotions is that these engineers are "lumpy" -- they are exceptional in some areas, but underdeveloped in others. For example, they may be very good at debugging the most complex problems, but they are not as good with finding alignment across teams.

I'd try to understand your manager's perception of you (along with others) to see what areas they think are holding you back most.

In a nutshell, this is a very good "problem" to have, and I think you should take your manager up on this opportunity. Here's why:

1. You are probably a solid E5 already - It has been 4+ months since the half started and promotions are lagging at Meta. If you weren't doing a solid MA at least, I'm sure your manager (who clearly cares about your growth) would have told you by now.
2. It is hard to fail - It is pretty hard to take E6 scope and do such a poor job with it that you're all the way back to showing E4 behavior. If you don't knock it out of the park, you'll probably still have enough impact/behavior to merit a solid E5 rating, and if you do, well, you're well on your path towards E6!
3. E6 scope is hard to find - There are so many teams at Meta where E6 scope just doesn't really exist. The fact that your manager is actively handing it to you is a golden opportunity - So many Meta E5s would love to be in your boat, haha.

And if I say no, should I be worried about my manager passing me up for potential opportunities in the future?

It's definitely sort of awkward. If your manager cares about you, there's a good chance that they don't want to overwhelm you, so saying "No" now may put them in an overly protective mindset where they don't bring up E6 opportunities proactively for you in the future.

All that being said, I recommend doing these 2 things tactically:

1. Share some of these thoughts with your manager - In particular, talk about how you feel thinking about failure modes. I can definitely see how an E6 opportunity could be scary for a fresh E5 - This is one of those "awkward" topics you should talk with your manager about.
2. Figure out your safety net - There is a big gap between E6 and E5, so if you take on this opportunity, there's a good chance you'll struggle here and there as you pick up E6 behaviors. In those scenarios, it's important to understand who has your back there - Maybe it's an E6 engineer or your team or a sister team. There's a good chance your manager already has some thoughts on this. Just ask them something like "So if things get really, really tough with this project and I'm having trouble handing it all, who/what can I count on in these situations?".