The base classes are also public by default in a struct. But, you're right--those are indeed the onty differences.
In "The Annotated C++ Reference Manual" by Stroustrup and Ellis (this was the language definition in 1990 just before ANSI and ISO developed standards), "class", "struct" and "union" keywords are "class keys" and all introduce class data types. (Unions can have methods and constructors, but cannot have base classes or be used as a base class and have a few other peculiarities.) So, the ability to do "OOP with structs" seems to have been in the language for a long time.
In what I've read from Bjarne Stroustrup (the creator of C++), the language was intended to enhance C rather than replace it. So, like oops, who posted before, I think that "class" was added to introduce new features and struct was retained for compatibility with C. The similarity in features may or may not have grown since the origins of C++ in the early 80s, but the final form was there by 1990. The place to look for an authoritative answer to this is in "The Design and Evolution of C++" by Bjarne Stroustrup. Unfortunately, I don't have it at all, much less handy.
The difference in defaults isn't a small issue, by the way. For OOP encapsulation, you really do want everything hidden that isn't explicitly declared public. However, structs must have it the other way for C compatibility--and that's no small thing either. Here's a Stroustrup quote from a 2000 Q&A on SlashDot:
---begin quote---
Let me just mention something I wouldn't have done differently: compatibility. Had C not been there to be compatible with, I'd have chosen compatibility with some another language. Innovation should focus on improvements and what works should be left as unchanged as possible. That way, people keep their existing tools and techniques and can develop from a base that is functionally complete. Also it saves the effort to re-invent the wheel and to teach "new" stuff that is equivalent to old stuff. Thus, C++ is as close to C as possible - but no closer.
---end quote---