There are two likely culprits. Number one is the belt itself. It may not fit, either because it's just the wrong size or because it isn't contoured properly. You don't say what type of pack you have, but often the belt is interchangeable. You can swap it out for a different size from the pack's manufacturer, or retrofit another maker's belt to it. So if the belt comes off, maybe try that.
Number two is your clothing. Particularly if you're wearing cotton shorts or pants, moisture may be getting trapped under the belt, and wet cotton clothes can chafe badly. So wear synthetic briefs made from Coolmax and shorts or pants from nylon or one of the newer performance fabrics such as Schoeller Dryskin. Some companies make shorts specifically designed for wear under a pack belt. An example: Mountain Hardwear's Synergy Short ($78). For one thing, it's made of wicking, fast-drying fabric so you're skin will stay dry. For another, it's cut specifically to avoid pinching under a pack belt, with what Mountain Hardwear calls a "conical cut" waist. So you might give something like that a try, too.